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[57] ABSTRACT 

A local host data processing system operating under the 
control of a local host operating system includes compo- 
nents of multiple emulating hosted operating systems. The 
host operating system further include a TCP/IP network 
protocol stack which couples to the communications facili- 
ties of the host system connected to a local area network for 
communicating with a number of remote host systems. Host 
and hosted operating systems share the same TCP/IP net- 
work protocol stack. A virtual network mechanism is con- 
figured within the local host system to be operatively 
coupled to the host network protocol stack and provide 
access to well-known port application programs. When so 
configured, the mechanism functions as another LAN to 
which multiple virtual host systems are attached for execut- 
ing applications under control of the emulating hosted 
operating systems. The mechanism transforms the well- 
known port identifier of each inbound packet into a non- 
well-known port identifier in addition to other station 
address identifier fields. & then redirects the transformed 
packet back to the JP layer of the stack for transfer to the 
appropriate well-known port application program being run 
by the hosted operating system of the particular virtual host 
system. The mechanism reverses this operation for each 
reply packet which it redirects back to the IP layer for 
forwarding to the remote system. This eliminates the need to 
specify additional protocol stacks and to provide additional 
communication hardware facilities for handling multiple 
instances of well-known port applications programs running 
on the different virtual host/multiple hosted operating sys- 
tems. 

20 Claims, 15 Drawing Sheets 
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struct 
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struct 
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struct mbuf *ifq_tail; 




int ofq_q;em; 




int ifq_maxlen; 




int ifq_drops; 



/* virtual host name, e.g. "veO", "vel", i, ve2" 
or "ve3" */ 

/* integer, 0-3 used to locate the 

ve softc structure */ 

/* maximum transmission unit*/ 

/* up/down, broadcast, etc. */ 

/* time til ifwatchdog timer called*/ 

/* routing metric (external only) */ 

/•linked list of addresses per if*/ 



r if_snd; 
/* procedure handles */ 

int (*if_init) 0; 
int (*if_output) 0 
int (*if_start) () 
int (*if_done) () 
int (*if_ioctJ) 0 
int (*if_reset) 0 
int (*if_watchdog) () 
/*generic interface statistics */ 
int ifipackets; 
int ifjerrors; 
int if_opackets; 
int ifcollisions; 
/*end statistics */ 

struct ifhet *if_next; 
u_char iftype; 
u_char if addrlen; 
uchar if hdrlen; 
u_char if index; 



/*output queue*/ 

/* init routine */ 

/* output routine (enqueue) */ 
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/* ioctl routine */ 
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/* packets sent on interface */ 
/* collisions on csma interfaces */ 
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VIRTUAL LOCAL AREA NETWORK WELL- address of the host computer on which the server resides, it 

KNOWN PORT ROUTING MECHANISM can contact that host. But, the client process must stfll have 

FOR MULT-EMULATORS IN AN OPEN some way of identifying that particular server process. 

SYSTEM ENVIRONMENT To solve this problem, the TCP protocol defined a group 

5 of well-known ports or well-known addresses which identify 

This is a continuation-in-part of patent application Ser. the well-known services that a host computer can provide. 

No. 08/473.476. filed on Jun. 7. 1995. now U.S. Pat No. For example, most TCP/IP implementations provide a file 

5.636371. issued on Jun. 3. 1997. entitled. "Virtual Network transfer server named FTP that a client process can utilize to 

Mechanism to Access Well Known Port Application Pro- transfer a file via a network to another computer system. The 

grams Running on a Single Host System", invented by Kin io 16 bit integer port established for FTP is 21 (decimal). Thus. 

C. Yu, U.S. Pat No. 5.636.371. evcry TCP/IP implementation that supports FTP. must 

RELATED PATENT APPLICATIONS "Z^J^Z ^ " * 

While this solved the problem of identifying well-known 

1. The patent application of Richard S. Bianchi, Dennis R. services, the utilization of this convention creates problems 
Flynn, Marcia T. Fogelgren, Richard A. Lemay. Mary E. 1 where a computer system which implements TCP/IP and 
Toyell and William E. Woods entitled, "Executing Programs supports FTP is required to run multiple well-known port 
of a First System on a Second System," filed on Sep. 28. application programs associated with different operating 
1993 bearing Ser. No. 08/128.456 which is assigned to the systems components which share a common host commu- 
same assignee as this patent application. nications protocol stack. Here, the well-known application 

2. The patent application of Kin C Yu and John L. Curlcy programs associated with the different operating system 
entitled. "Sockets Application Program Mechanism for Pro- components, such as those of an emulator and host system 
prietary Based Application Programs Running in an Emu- botn rc( l uircd to utilize the same identical well-known 
lation Environment," filed on Mar. 30. 1995, bearing Ser. P™" in identifying like application program services. This 
No. 08/413333 which is assigned to the same assignee as & P ves rise to a naming conflict between the different appbV 
this patent application. cation program services. 

Relative to problems relating to process migration, one 
BACKGROUND OF THE INVENTION author has observed that support for process migration is a 
1 Field of f 1st characteristic that is increasingly important Protocols such 
~_ 3. as OSJ, X.25 and TCP/IP that use such machine addresses to 
The present invention generally relates to methods and identify processes make migration difficult because a pro- 
mechanisms for conducting internetwork communications. cess cannot take its address with it when it moves. The 
More particularly, the present invention relates to methods author describes the use of a new custom protocol called a 
and mechanisms used by a computer system which executes Fast Local Internet Protocol (FLIP) and an architecture 
application programs originally developed to run on another which permits servers to migrate to new machines without 
computer system and provides network facilities to carry out requiring any manual reconfiguration, such as TCP/TP 
communications over a network with other computer sys- requires. For further information regarding this protocol. 
tems - reference may be made to a section 14.5 entitled. "Commu- 
2. Related Art ni cation in Amoeba" of the text entitled. "Modern Operating 
With the advent of open system platforms which operate 40 Systems" by Andrew S. Tanenbaum. published by Prentice- 
under the control of versions of the UNDC operating system. mc - Copyright 1992. One problem noted relative to 
it becomes more and more desirable to be able to efficiently ^ solution is that the new protocol requires considerable 
run application programs developed for earlier computer changes to be made to a host system. Hence, this approach 
systems, such as proprietary based systems on such open ' s not practical where it is essential mat the host computer 
systems without having to rewrite or port such application 45 operating system remain intact 

programs. A computer system which accommodates such Another approach which has been considered is to pro- 
application programs is described in related copending vide duplicate communication facilities wherein a separate 
patent application of Richard S. Bianchi. Dennis R. Flynn, TCP/TP protocol stack and separate hardware facilities are 
Marcia T. Fogelgren. Richard A. Lemay. Mary E. Tovell and provided for servicing the network demands of two distinct 
William E. Woods entitled. "Executing Programs of a First so sets of well-known port application programs. While this 
System on a Second System." solution may be satisfactory in terms of eliminating the 
Generally, such application programs are required to naming conflict, it would create considerable processing 
operate in conjunction with and communicate with other delays causing application programs executing under con- 
computer systems over internetworks. Many of these com- °f an emulator to run too slow resulting in decreased 
puter systems utilized standard communication network 55 overall system performance. Also, this approach is too costly 
protocols, such as TCP/IP. which are normally implemented m te rms of system resources and is unable to take direct 
as part of the computer system's operating system (Le.. advantage of existing host facilities, 
kernel). Also, such computer systems generally support Also, it becomes advantageous to provide support for 
multiuser environments in which it was possible for more different interface protocols or hardware interfaces, espe- 
than one user process at a time to be using such networking eo cialry in the case of emulating environments. Here, it has 
facilities. To implement mis. the communication protocol been the practice to provide multiple protocol <a«ry* which 
implementation required the adoption of a method for iden- enable the use of such different protocols or different hard- 
tifying the data associated with each user process. That is. ware interfaces. 

when a client process wanted to contact a server process. Accordingly, it is a primary object of the present invention 

such as FTP or Telenet, the client process must have a way 63 to provide a method and system which enables application 

of identifying the server process that it wants to use. In programs running in under control of multiple instances of 

TCP/IP. if the client process knows the 32-bit Internet different operating system components sharing a common 
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forT^Sfn^r^ s f ck, 1 ottti,ize w*n-toK>wn port, the routines of the virtual network median used in 

for idenufymg Uke protocol application program services. processing client requests to be shiurT^T muMpk 

It is another object of the present invention to provide a virtual host systems, 

method and system for executing application programs The virtual network mechanism contains a mapping com- 
T^t ° c ^ mca *° ns Pro^l stack to s ponct which maps the different IP address porton^ 

utilize weU-known port addresses for designating well- predetermined manner. The mechanism toSlc^ 

known application programs accessible by client application the packet containing the mapped IP addresTonto fee 

programs on a remote host system which is transparent to the interface of the IP module justa/ifThad been received from 

remote system and requires minimal change to the host the other network. In greater detail, l^StoS 

tauung of such application programs. a specific hosted system and to replace oJ^cO^Zn" 

SUMMARY OF THE INVENTION P ° lt . numba " non-wcll-knowo port identifier of the 

_ . services application program/server (e.g. FTP application 

me above and other objects of the present invention are server). Additionally, the mapping unit substitutes a virtual 

achieved in a preferred embodiment of the virtual network 15 host address for the IP source address of the requesting client 

mechanism of the present invention which operates under apphcation program on the remote host system so that any 

the control of a host operating system, as for example, an rc &y packets provided by the application services server in 

enhanced version of the UNIX operating system running on response to the request are automatically directed back to the 

a local host computer system which connects to a local area virtual network mechanism. 

( ^J 5r vi? t f nietWOrk fOT TOmmunicatin g ^ a 20 For each reply packet received, the mechanism 

l!^ e , h ° S l Sy !!! mS * standard eanuntui- substitutes/restores the appropriate IP source and destination 

cations protocol In the preferred embodiment, the host address portions in the IP address and reintroduces the 

system also includes the components of a plurality of hosted packet onto the network interface as if it had been received 

operating system components, such as for example, multiple _ from the other network The IP stack layer now directs the 

instances of an emulator. * reply packets ^ £ 

Inc host operating system further includes a communi- gram on the remote host computer in a transparent manner. 

cations network protocol stack which in the preferred This ensures that the sharing of the host system communi- 

embodiment corresponds to a host TCP/IP protocol stack carton protocol stack remains completely undetectable to 

Both each hosted and host application programs share the client programs running on the remote system, 

single protocol stack. The virtual network mechanism of the 30 The present invention processes client requests for a 

rE^T*^ ^ 68 * c t nam | n 8 ^ plurality of virtual host systems while elinunating the need 

the use of multiple instances of well-known port application to communicate through additional protocol stacks or to 

programs being run by each hosted and host operating provide additional communication hardware facilities. This 

, "J 8- , , 33 in turn enhances overall system performance as well as 

in me preferred embodiment, each remote host computer eliminating the need for having to allocate additional system 

system which communicates with the host system of the resources (eg. memory). 

SS^ I i DVCn ^ n ™ inta ? ctwo * is configured either While the preferred embodiment of the present invention 

statically or dynamicaUy to have the local host system is described in terms of an emulator ^vironment. breach 

^°HW^, 8 ^ ay J ah r t ^ mthatCOnncCtStwoar « can be generally applied to systems which share a single 

r^Jf ? mC bOSt System C2USCS P ro,oco1 *** °* tbc same host system. For ot^e 2 

Z^IT "T? *? lnternrtwark (heterogeneous be deskable to have multiple processing units ™ afferent 

to ; anofl ? £I DCtW0 *" accor ^ of * e ope«ting system f*T££ 

JZJ? identifier information contained in the protocol stack. Also, it may be equally desirable to have 
ixrworK address. 4 , different operating systems running on the same host system 

me mechanism of the present invention is configured share the same protocol stack, 

wittun the host operating system as a separate network Also, it will be noted that the teachings of the present 

interface wtuch couples to the network protocol stack just as invention are not limited to requiring thatfteotte Sri 

another physical network." This allows the mechanism to or party to the cxHnmumcatic^rtypfcllly an 

^ZfJtZ ^ d lntanCtWork finality 50 program, be located in a fA^y sq^^fnputo 

assooated with .mt .communication networks. The IP layer system. The communications could tie p^bcr^eTm^ 

* 1"* t0 hOStCd ^ to h0St "*» Md 00C ° f P»^ity of hosted ^e^ or 

the virtual network mechanism as if it were another network between two hosted systems 

iut™ .„ ,. ^ . when taken in conjunction with the accompanying drawines. 

More specifically, the virtual network mechanism utilizes u^yiug urawings. 

a different set of control data structures corresponding to a BRIEF DESCRIPTION OF THE DRAWINGS 

different one of the virtual host systems. Each set of struc- FIGS, la and lb illustrate in block diagram form, a host 

tures includes an interface network structure used for con- 60 system which incorporates the method and apparatus of the 

necting the virtual network mrrtianisrn to the network present invention. 

^ , S l . C0 . IlEral saucb ^ e which ^presents the FIG. 2 is a simplified system Mock diagram illustrating 

particular vutual host system and a diem table structure the use of the virtual network of the ,JeSS 

which is used to process client requests directed to the internetwork 

^L*? 3 rem ° tC !i l0Cated dfcm SyStenL ^ 65 FIG. 3 is a diagram fllustrating the positioning of the 

-rfefZfi™ *"* ° f J auaa ™ » °P=^ ^ a virtual network SL wkrTf ifyercTc^m^nicatSn 

corresponding number of vmual host systems, this enables network, according to the teachings of the prese^ven^ 
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• FlQ \ 4 .^ 3 blOCk , dia S ram of the virtual network mecha- pointers to interrupt save areas for storing hardware param- 

nism of the present invenuoo. cters related to the task The indirect request bl«* (KB) 

FIG. 5 illustrates in greater detail, the different structures structure 36 contains information defining the operation 

utilized by the virtual network mechanism of the present requested by an associated task and includes pointers iden- 

invcntion. s tifying the task and its associated task control block (TCB) 

FIGS. 6. 7a through 7g and 8 are flow diagrams and 311(1 a pointer to the associated IORB structure, 
associated data structures used in describing the operation of 'UNIX is a registered trademark in the United States and 
the present invention. other countries, licensed exclusively through X/Open Com- 
pany I imfti-rf 

DESCRIPTION OF THE PREFERRED io ,* ATSC u arr - rt ... . „ . . ,„ . 

Bunnrmrewr "ADC is a registered trademark of International Business 

EMBODIMENT Machines Corporation. 

FIGS, la and U> collectively constitute a block diagram The input/output request block (IORB) structure 38 is 

of a host system 54 which incorporates the virtual network used as the standard means of requesting a physical VO 

mechanism of the present invention. As shown, the system service. 11 contains information such as a logical resource 

54 includes a hardware platform 56 which contains the number CLRN) that identifies the UO device being 

hardware elements such as a central processing unit 58a. a addressed as well as the location and size of the buffer to be 

main memory 58* and a number of input/output peripheral used for the transfer and the specific function (operation) 

devices 58c and a communications facility such as an requested. The resource control table (RCT) structure 40 

Ethernet local area network (LAN) 58d* for connecting contains information describing the resources, such as its 

system 54 to other processing systems via standard com- characteristics or information regarding the tasks or requests 

munication network facilities. being executed by a corresponding resource as well as 

The central processing unit (CPU) represented by block pointers to its associated task control block (TCB) structure. 

58o is a reduced instruction set (RISC) based processing unit Additionally, two other structures depicted in FIG la are 
which takes the form of the RS6000 microprocessor manu- ^ a group control block (GCB) structure and a user control 

factured by IBM Corporation. As seen from FIG. 1, hard- block structure of block (UCB) 29. The GCB structure 

ware platform including processing unit 58a operates under contains information required to define and control the 

the control of an enhanced version of the UNIX* operating operations of a specific task group which defines a named set 

system such as the ATX*» operating system. Portions of of one or more tasks with a common set of resources within 
physical memory represented by MEM block 586 are Olus- x which a user and system function must operate. Each group 

trated in terms of (he layered construction. As shown, has a two character name (eg.. $L. $S) by which the group 

memory is divided into two basic levels, a user level and an is uniquely known to the system. The GCB structure 

operating system level. The user level is divided into emu- includes information identifying (he lead task whose execu- 

lated system (ES) and host shared memory space and host or tion spawns all other tasks required for executing group 
an operating system kernel native memory space. The shared 35 programs. As indicated, the GCB structure includes a num- 

roemory space contains the ES executive level 16 which ber of user control blocks CUCB). each of which contains 

includes a plurality of executive program tasks 30 spawned information defining the user's personality such as user node 

by ES executive services components of block 28 for identification, user group id within a node, user task id 

executing ES TCP services application programs/servers 22 within group, user person id and pointer information to 
and system administrator programs 24. ^ directories to which the user has access. 

In the preferred embodiment, the well known pan appli- As shown, the emulated system utilizes a further data 
cation programs, such as for example. TCP application structure corresponding to system control block (SCB) 
programs provide FTP and Telenet services to client pro- structure 27. This data structure is created at system startup 
grams. As well-known in the art telenet service application and contains information defining system resources and 
program allows an interactive user on a client system to start 45 pointers to the different task groups established by the 
a login session on a remote system wherein the client system represented by a corresponding number of group 
process passes the user's keystrokes to the server process on control blocks in the system. For further information regard- 
the remote system. The FTP services application program ing such structures and their relationships to each other, 
permits the transfer of files from one system to another and reference may be made to U.S. Pat No. 5.1 11 384 and the 
provides a rich set of features and options, such as user » publication entitled, "HVS PLUS Systems Concepts" pub- 
authentication, data conversion, directory listings, etc. In lished by Bull HN Information Systems Inc., Order No 
operation, the interactive user invokes an FTP client process HE03-01. 

^ , o C „ 1 ^iT , ^» TbC CUCDt prOCCSS r tabUsh " * ron - As indicated in FIG. lb. the shared memory space further 

J* %Z $aVa P"*** °° * C remote sy^ 6 ™ deludes a memory queued interface (MQI) represented by 

r^f,7„^ r T ^ to™**™ * block 84 wi^ovides a form of intc^oc^^rcmum- 

between the cbent and server processes, one for control cation mechanism and a software active queue (SAO) of 

^formation and the other fa die data being transferred. The block 88. SAQ block 88 represents a data structure used to 

^^r^'nT 11 ^ a «^^°™^°» on the provide the path by which the results of the operations 

remote system and the files then can be transferred in both performed by the kernel level components are passed back 

60 or returned by the host processes to the requesting emulated 

In the emulated system, each task 30 utilizes a plurality of system user level tasks 30 being executed. Thus, it can be 

data control structures, such as a task control block (TCB) viewed as functioning as an output stage of MQI 84. This 

structure 32. an indirect request block (TUB) structure 36. an data structure is similar to data structures which are used by 

input/output request block (IORB) structure 38 and a the emulated system operating system. 

re£0 ^^°°^ lable (K/^ 40. The task control 65 MQI block 84 is a semaphore data structure which takes 

block (TCB) structure 32 contains information pertaining to the form of a single linked list controlled by semaphores 

the state of execution of the associated task as well as through a set of routines which are executed by theTarious 
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host processes operating within different levels or layers bat 
want to communicate with each other. Its routines are used 
to manage queues within the pseudo device drivers 74 and 
the software active queue 88. 

Executive Services Components 28 5 
As seen in FIG. la. the executive services components 28 
of executive layer 16 includes a plurality of components or 
facilities which are equivalent to those facilities normally 
included in emulated system. The emulated system is a 
multiprogrammed multiprocessor system. The facilities 10 
illustrated in FIG. la include a listener module 280. a file 
management facility 282. a socket monitor call command 
handler unit 284. and an ES socket library 286 which are 
arranged as shown. The listener module 280 is responsible 
for monitoring the operations of terminals configured for is 
login and for initiating user tasks in response to user 
commands. As indicated in FIGS, la and lb. listener module 
280 runs as a task 30 with Its own set of unique data 
'structures. 

The listener module 280 is able to consult a profiles file 20 
containing user specific registration information such as user 
id. login id and password requirements tabulated by the 
system administrator for all registered users. The listener 
module 280 checks the user profile when monitoring the 
privileges and/or restrictions given to each user. The file 25 
management facility 282 includes the conventional shared 
data structures and set of routines normally provided to 
perform functions that access such data structures to control 
the synchronization of concurrent processes or tasks in 
addition to performing various system services or functions. 30 
That is. the facility responds to system service monitor calls 
identifying the types of services requested (e.g. creating or 
deleting files, reading or writing records or blocks in files) 
which result in the specified system services being executed 
by the emulated system on behalf of executing user appli- 35 
cation programs. 

A monitor call unit (not shown) receives monitor calls 
from the interpreter component 72 which are in turn to be 
executed interpretively using the ES executive service com- 
ponents of block 28. A command handler unit (not shown) 40 
contains the routines that respond to user commands entered 
via a terminal or program. In response to such commands, 
the command handler unit routines invoke the appropriate 
tasks for executing such commands. 

The ES components involved in handling socket opera- « 
tions include an ES socket command handler unit 284 and 
ES socket library 286. The ES socket library 286 is con- 
structed to provide the same socket application program 
interface (AFT) as provided in the ymniat^ system. This 
interface is described in detail in the manual entitled. 50 
"GCOS 6 HVS TCP/IP SOCKET API FOR C USERS," 
published by Bull HN Information Systems Inc., Copyright 
1993. Order No. RD89-00. 

The ES socket command handler unit 284 contains a 
plurality of routines which operate to convert HVS/ES 55 
socket calls into the appropriate low level request input/ 
output (RQIO) monitor calls accompanied by IORBs created 
by mapping/translating the socket library calls into the 
corresponding socket function codes. As described in detail 
herein, the IORBs are forwarded to the main socket server 60 
component by the EMCU via the MQI interface. The main 
socket server component then issues the a p pro p riate host 
(ATX) socket calls to the host system socket facilities. 

Emulator Level Layer 68 ^ 
As indicated in FIGS, la and lb. the next layer within the 
user level is the emulator executive level 68. This level 
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includes certain components present in the emulated system 
which have been transformed into new mechanisms which 
appear to the remaining unchanged components to operate 
as the original unchanged components of the emulated 
system. At the same time, these new mechanisms appear to 
me components of the kernel level 64 as native components 
with which the host system is accustomed to operate. As 
shown, the components include the interpreter 72, an emu- 
lator monitor call unit (EMCU) 73. dynamic server handler 
(DSH). main socket server component 98. a number of child 
socket processes 96 and a socket control table 94 operatrvely 
coupled together as shown. 

As indicated in FIG. la. the emulator executive level 68 
further includes a plurality of pseudo devices drivers 
(PSDD) 74 for each input/output device or type of input/ 
output device which is required to be emulated by host 
system 54. For example, the pseudo device drivers 74 win 
include PSDDs for terminals, disk drivers, tape drivers, 
displays and for certain communication devices. 

For a more detailed discussion of other aspects of the 
SAQ 88, MQI block 84. PSDD 74 and other emulator 
components, reference may be made to the related patent 
application. 

The interpreter 72 successively fetches the instructions of 
an emulated system application program, categorizes each 
instruction and executes it interpretively through sequences 
of RISC instructions which allows CPU 58a. MEM 58fc and 
other elements of host system 54 to emulate the operations 
of corresponding elements of the emulated system. The 
interpreter 72 includes a monitor call (MCI.) table contain- 
ing information for each possible monitor call which it 
utilizes to determine whether to trap or send an ES monitor 
call to the ES executive services components 28 for execu- 
tion of the instruction or to make an emulator call to EMCU 
73 for execution of the instruction through the services of an 
appropriate C language routine (server). The EMCU 73 is 
responsible for acquiring from the host system 54. the 
necessary memory and other resources, for initializing the 
emul a te d system data structures and invoking interpreter 72 
and the various server processes. Both the interpreter 72 and 
EMCU 73 run as host processes. 

As viewed by the host system, the ES service components 
28 and tasks 30 being executed on behalf of the application 
programs, the interpreter 72 and EMCU 73 are executed in 
the system 54 of FIGS. In and lb as a single process 
(emulator) 80 wherein such process corresponds to one or 
more user processes as defined by the conventions of the 
host operating system being run on host system 54. Thus, it 
is possible to have multiple instances of the emulated system 
concurrently emulated on host system 54. 

The dynamic server handler (DSH) 92 is created by 
EMCU 73 during system initialization. The server 92 com- 
municates with emulated system processes through MQI 84 
as indicated in FIG. lb. The lower level main socket server 
98 and socket control table 94 are dynamically created by 
higher level server 92 for carrying socket operations accord- 
ing to the present invention. The main socket server 98 
creates child socket processes as a function of the type of 
socket operation to be performed and manages such child 
processes through socket control table 94. All of the servers 
operate as root and therefore have super user privileges with 
access to any file within the host system 54. The server 92 
includes mechani^im specifically designed for validating 
security at the user level in conjunction with the execution 
of dual decor commands and functions. 

For the purpose of the present invention, the components 
92 through 98 collectively can be viewed as a socket server 



09/03/2003, EAST Version: 1.04.0000 



5,734,865 

9 10 

i^T^Lf, ~ mmunicate <"« host how the VNET mechanism 100 of the present invention is 

system socket layer ft wfll also be noted that toe level 62 incorporated into me host system of HG1. As seen from fee 
also includes the different host TCP application service Figure, only the components relevant to describing the 
programs 75 which provide TCP and Telnet services. These teachings of the present invention are depicted in FIG. 2. As 
application services programs/servers are represented by s indicated, the VNBT mechanism 100 functionally represents 
block 75 in FIG. 1* and also communicate over the same a plurality of virtual host systems veO through ve3 running 
host system socket layer and sh are the same TCP/IP network a corresponding number of emulating hosted operating 
protocol stack facility 99. systems, such as emulator 80. In the preferred embodiment 

each virtual host system connects to a local area network 
Operating System/Kernel Level 1Q which corresponds to the virtual LAN of block 100. As 

The operating system/kernel level 64 includes the stan- herdn ' 016 netWQrk structure of the emulated 

dard mechanisms and components normally included within systcm tams 01 ^ addrcss is incorporated into the host 
the host operating system. As shown, level 64 includes a 54 by configuring the virtual network mechanism 

kernel process manager component 70 and a number of host mt ° mC boSt ^stcrn as described herein, 

kernel I/O services (KIOS) processes 66 for each pseudo 13 . As snown - me mechanism 100 includes a virtual network 
device driver (PSDD) 74 which is to be emulated by the host interface portion 100-2. In many respects, this interface is 
system. Additionally, in the preferred embodiment of host functionally similar to the network interface labeled SSd 
systcm 54. level 64 is assumed to contain the standard utility connected to the physical local area network (LAN) 18. In 
programs, shell, editors, compilers, etc. and libraries (eg addition to the LAN. the interface SSd includes the standard 
I/O libraries, open, dose) which are accessed in the host user 20 software routines (e.g. drivers) which provide a uniform 
mode. For further information regarding the use of such ""erface to «he Internet Protocol (TP) network layer. Thus, 
arrangements, reference may be made to publications of the me mterface performs all of the necessary communications 
IBM Corporation describing the ATX operating system. between the IP layer and the physical LAN normally through 

In the preferred embodiment, the kernel/operating system appr ° P ™! le Physical device handler. For the purposes of 

level 64 further includes as an uiterprc^ss comrnunicatioos 23 f I™*"™*"*, software portion of the network 
facility, an implementation of "sockets" which included EgJ" r^^^V^- f °™ « t^J**™* 
host sockets library 97 for storing a plurality of socket 806 described tn standard IBM publications, 

subroutines and network library subroutines and a TCP/IP described iater herein, die virtual network interface 

network protocol stack facility 99 arranged as shown. The 1 , 00 " 2 is ^ constructed to incorporate the same fiiDction- 
stack facility 99 connects to network driver software (eg 30 V ** lnduded "» *e network interface software of block 
Ethernet. Token-Ring. FDDI) included within kernel level 5Sd -}n the case of an Ethernet LAN consisting of host 
64 (not shown) which communicates with the Ethernet/ machines which use the TCP/IP protocols, such as shown in 
Token-Ring/FDDI LAN SSd nG - ^ mere m two t yP es °f addresses. One is the 32 bit 

As indicated in the wh-m nf Pir ij, .„ Internet address and the other is the 48 bit Ethernet address. 

TCP/IP. These are reliable stream delivery. coXtionless £ " (AR f ) "T* 1 ' * " Itowed t0 

datagram delivery and raw socket deKverv The^rf^ 40 a s ^ aal P* 0 * 64 ° n Ethernet that asks the host 

™^Hm»^T.Z; »5{.kV 77- ^' P r = etre d with a specified IP addrcss to respond with its Ethernet 

embodiment uses reliable stream delivery carnmumcation arfrfrrc* Th* w.^<^-. n . " 

services. For further information regarding sockets refer _ ^ broadcashng host system then can store the 
ence may be made to various weU-tao^ubSo'nTaTd ZTr^^^^ti^l^f -^ 
texts such aspublications of IBM Corporation describing^ 4 , *1 £ aSeT ***** 
ATX Version 3.2 for RISC System/6000 and the text entitled 45 ZZlZT , ™, , , 

"UNTX System V Release 4: An Introduction for New and *• Sff avs - «c the layer/module that handles 

Experienced Users," published by Osbom McGrawHifl. !L "T* mt TT?^ W Uya providcs a 
Copyright 1990 by American Telephone and Telephoned L ^^i 1 * connec - 

Telegraph Company. ^ tionless because it considers each IP packet independent of 

so all others. Any association between packets is provided by 
Virtual Network Mechanism uc upper TCP layer. Every IP packet contains the source 

. ,. address and destination address as discussed herein so that 

According to the teachings of the present invention, the each packet can be delivered and routed independently. The 
operating system level 64 also includes a virtual network IP layer is unreliable because it does not guarantee mat IP 

T^^n^^^,^ W ^T T f VCl L COllplCS 10 thc 55 P*<*e* ever ct delivered or that tbey are delivered correctly. 

TCP/IP network pntocoittK* facility 99 in the same The IP layer computes and verifies its checksum. This allows 

SZl^?^ ^kmterfacea^sociatedwimme network it to verify that the fields that it needs to examine and 

dnver and LAN couples to faculty 99 as explained in process. When an IP header is found in error, it is discarded. 

^ J^T - C TJ? mechamsm 1M 3150 C0U P les 10 « with the assumption that a higher layer protocol will retrans- 

°kJ!T 1 structures represented by block 102 « mit the packet If the IP packets arrive at a host or gateway 

Seated in host system memory which are used to process so fast that they are discarded, the IP module sends an 

chent requcsto received via faculty 99 directed to a plurality Internet Control Message Protocol (ICMP) source quench 

of virtual host systems/hosted systems. message to the original source irJorming mat system that the 

FIG. 2-Simplified Network Block Diagram date is arriving too fast. 

pttj „;„,„,;,= . . , ,. , &3 The present invention makes use of the routing capabili- 

inZL^l ^T™ rf 8 P ° rtion ° f a ties of the IP module. A gateway determines the roureTa 

internetwork system 10 which discloses in greater detail. packet by consulting a network routing table. In TCP/TP. 
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routing can be one of two types. The first type is static (process) is defined by the IP address of the host system on 
routing which uses manual input to update the routing table. . which it runs and the port number through which it com- 
The second type is dynamic routing which uses routing municates with TCP/TP. Sockets are used to establish corn- 
daemons to update die routing table automatically when new munications. A socket is the pair of numbers which uniquely 
information is received. Therefore, when the host system 20 5 identifies each application. More specifically, a socket is 
desires to communicate with the virtual network mechanism defined by an IP address and port number. As discussed 
100. it utilizes a route command which allows a user on host above, the Telnet and FTP application programs use the 
system 20 to make manual entries into the network routing same port number in all TCP/IP implementations. Those 
tahles. In the preferred embodiment, a host system route "assigned" port numbers are called "well-known ports" and 
command is used to statically configure a gateway for the 10 the standard application programs are called "well-known 
virtual host system 100-4 connected to the virtual LAN of services." Thus, the socket layer is said to support the 
virtual network mechanism 100 to which the user on host concept of reserved ports in the Internet domain wherein 
system 20 wants to connect The route command has the standard Internet application programs are assigned well- 
following format: route add -net network_address known ports. 

*?r,^!f" S ^ n *«°^ 8 ^^ re ^ te4 15 TCP or transport laycrprovides a connection oriented 
fi \ C °f gUred ? a ^ abC " reliable full duple^byte stream service to an application 
permanent configuration, gateways can be configured via the program. The TCP module contains the nccessarylogic to 
operating system configuration management system. provide a reliable virtual circuit for a user proceLtltSes 
As shown in FIG. 2. the IAN 18 in addition to connecting the establishment and termination of connections between 
to host system 54 also connects to another host system 20. 20 processes, the sequencing of data that might be received out 
When the virtual network mechanism 100 is configured into of order, the end to end reliability (checksums positive 
the system, it is viewed by the host system 54 as another acknowledgments, timeouts) and the end to end flow con- 
network since it is constructed to have its own separate troL TCP uses 16 bit integer port numbers for identifying the 
network interface. Each IP address includes a network ID data associated with each user process 

^i^^ 051 ^ AS at T- h0St ^^Sf Whkh 25 As discussed above, the IP layer provides the packet 

^vhTc^T* ^ T-yjT^ ^ t ***** servicefortheTCP layer ^computes and vS 
S3 £1 h £ ? network interfaces, one for each its checksum. The IP layer uses 32-bit integer IP addressed 
network with which its communicate regardless of network for identifying the networks and host oomputerTon &e 
yp ^ internet. 
A gateway receives packets from other hosts and gate- 30 aetwork interface ^ frame9 fl { _ 

route rl£t72L „t ^f ^T^tf: caUy connected hosts and" i/ responsible roTh^Sa 
route packets from one network to another. Since each IP , rr ,.„ . . , ^ 

address includes a network ID and a host ID, gateways can IZt^^'J^^r ^ ^ pr ° VldeS me 

easily extract the network ID field from the ^addre£ Z tl^a^ ^e^l^^T^^ T 

route IP packets based solely on the network ID. Since 35 me^^V^, f ^nf ?k. T ? l^J* ^ 
~Z. i- . . .. ~~ , . rnented to conform to one of the physical networks, such as 

Ti u deSUnati ° n " ,4 for example. Ethernet LAN requirements and are hence 

not according to destination host, a gateway need only to i^ le Z*h the prefix "EtherneT" As indicated, th^e lay^ 

,t S° n ^ h ne 7 OTkS - aDd d0e ? "°^to could r^de ^conform to Token-Ring otTOD?* I™ 

know the location of every host system on an internetwork. as other types of physical networks. 

Thus, the destination network takes care of sending the 40 'KT. ... " "_~ ' . a 

packet to the destination host mG 3 illustrates the type of data flow taking place 

, , , ,„ , , , . , between the different layers. More specifically, the figure 

Therefore, when host^stem 20 adds the virtual network shows the addition of control (headcrSrmatiorterSed 

address to its network routing table, the same routing encapsulation, by the different layer modules whendata 

infection is also passed to host system 54 through static 4J ^ xnt by a TCP application ^ogramlc ano«£r host 

or dynamic routing and entered into the network routing system, 
tables utilized by the IP module of the host system 54 on 

which the virtual network mechanism 100 resides. FIG. 4-Wefl-Known Port Virtual Network 

Accordingly, as described later herein, the IP module auto- Mechanism Block Diagram 

matically routes those IP packets/designating the virtual ,„ mr> a •» - ^ .„ 

LAN to virtual network mechanism 190 50 4 *? vano " s P*ts of the Virtual Network 

Mechanism 100. As shown, the mwhanicin 100 includes the 

FIG. 3- Virtual Network Mechanism Location components 100-2 through 100-14 which operalivcly con- 

, n , _ . . „ .. nect as shown. The IP interface component block 100-2 

FIG. 3 illustrates in diagrammatic form, the positioning of represents the various interface routines utilized by the 

the virtual network mechanism 100 according to the present 55 different sets of structures correspondin K to the virtual host 

invention, relative to the TCP/IP conceptual layered orga- systems veO through ve3. In thTpreferred embodiment the 

nizanon. As indicated in FIG. 3. the VNET mechanism 100 interface table structure 100-2 defines one of the three types 

directly couples to the IP layer so that it looks like another of physical interfaces. For the purpose of the present 

network interface to the host operating system TCP/IP invention, the interface 100-2 conforms to the type of 

r^OOTlstacfcThear^cationUyeristhelevelatwhichthe so network interface utilized within the ATX operating system. 

TCP/IP appliranon programs or user processes operate/ Generally, this type of interface accepts output packet of a 

^ ^!^ Vera f ^cation programs provided by almost specified maximum length, and provides input packets 

every TCP/TP implementatiaii include FTP and Telnet which received from its medium to higher level routines 

were discussed above. ~ 

The socket layer is the first kernel layer and it provides an 65 Control Data Structures 

application program interface (API) to the TCP/IP commu- As explained herein, each virtual host system is rcpre- 

nications protocol. Each TCP/TP application program sented by a set of control data structures which include an 
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ifnet structure, an ve_softc structure, and client table struc- cated in FIG. 5. The tcp_state field defines the virtual 

hire. The ifnet structure for the network interface defines a operational state of the client tabic relative to processing a 

queue or network interface table for such interface which is given client request by the TCP module. The dient_flags 

used by the IP module routing software code to locate the &M is used for storing information pertaining to the state of 
interface. It contains control information defining the type of 5 me table entry (eg. availaWe=CLIENT_EMPTY==O0 in 

!f S if ? ro P? ieS- routmcs and statas statistics as use=CLIENT_JNUSE=0. closing=CLIENT_ENDING= 

desm^heran below. 02). The dienl IP address field is used for storing the client 

TT,?fiffJ TTZ ^ m ^ .""I 10 - ? !i ff a<Urcss « the client tcp dst port field is used for 

The ftincuons of the ifnet sh^ctore include loading and storing ^ dientTcp sourcc aui ^ z _ ^ cUeo , t to 

initializing. comrnuiucarinE with the IP network layer, com- „ _ c , . . . , . ,.T~T r\ * 

mu3g with device handler software, translating an IP 10 ^^V^t J^, *V? m' TCP deshnahon port 

address to a hardware address for the underlying device number - Usflv ; *" mer . ^ fieW 15 for a 

driver software, handling ifnet specific ioctl calls and ter- ^ valuc "Seating A* number of minutes which 

ruinating and unloading The present invention makes use of **** ela P scd smcc merc was a client request from the 

this same type of network structure mechanism utilized by particular remote client process. This used to remove entries 

the host operating system for a physical network interface 15 assigned to client processes which have been rendered 

unit which eliminates the need to introduce any additional inactive. 

network structures or software to be associated with the Continuing on with the description of FIG. 4. it is seen 
virtual network mechanism 100. that incoming packets are applied to an input receive cam- 
As indicated in FIG. 7c, the ifnet structure contains a ponent 100-6 which determines the type (i.e.. ICMPorTCP 
number of different fields, only some of which are utilized a> protocol type message) and the source of packet message 
by the virtual network mechanism 100. A first field is a name being received and forwards it to the appropriate component 
field (if_name) which identifies the virtual host ( i.c, veO, for processing. More specifically, if the packet is an ICMP 
vel. ve2 or ve3). A second field is a unit field (if_unit) message packet such as an echo message used by the 
which is an integer used to locate the virtual host system internet Control Message Protocol, it is forwarded to ICMP 
control strurture associated with the virtual host system (i.e. is ech 0 processing component 100-16. If the packet is an 
f^^Jhfi "if* 3 " flsoinclades into- Ethernet, Token Ring or FDDI type message>cket. it is 

l??? e JT fiC l^-L a l S) forwarded to either inbound component 1004 o^urbound 
is used to indicate the state of the interface/virtual host „„„„ . 1(w , _ , . ^7 .. . . " 

system (e.g. an EFF_UP state indicating that the interface/ ^ 0mp0Q "? funcdon °f^.ch source originated 

virtual host is up. an EFF_RUNNING state indicating that „ me 71,5 ICMP com P onent 1S included m order to 

the interface/virtual host is running which allocates res Pond «> P"»S inquiries. 

resources), an ifaddr structure which contains information The component 100-8 processes inbound tcp packets 
about one interface address which is a pointer to a linked list originated from a remote host system while outbound com- 

of addresses used by the IP module to locate all of the ponent 100-12 processes outbound tcp packets originated 

network interfaces of a given address family on the host „ from the virtual local host system. As indicated, the inbound 

system (e.g. Ethernet interface SSd), interface routines fields component 100-8 contains the routines of block 100-8a 

which identify the different routines used by an attached which save flic packet IP address. TCP source and desu'na- 

interface (e.g. if _init, if_output, if_Joctl) and interface tion P 01 * numbers. It also includes the routines of block 

statistics fields. 10#-8i which create a set of mapped TCP source and 

FIG. 5 illustrates the set of control structures used by each 40 dcstination P 0 * 8 according to <ne present invention which 
of the virtual host systems ve0-ve3. Each control data are used to reformat the IP address and TCP ports resulting 
structure designated ve _softc defines a different one of the m forwardin g * e packet to the appropriate emulated system 
virtual host systems (i.e. veO through re3 of FIG. 2). As TCP application program (eg. ftp. telenet, etc.). The out- 
indicated in FIG. 5, each ve_softc control structure also bouaA component 100-12 contains the routines of block 
designates the client table structures used by its associated AS 100 ~ L2a which retrieve me appropriate previously stored 
virtual host system to process requests received from original remote host IP address and TCP source and desti- 
remotely located client processes nation port values. These values are used by the routines of 

As seen from FIG. 5. each ve_lsoftc structure includes a bl< ** V**" 12 * * reformat me P*** for rerouting the 

number of different fields and structure designated struct padcet "»* to *" remo,e 1,051 svstem 20 

arpcom through virtual IP address. The structure arpcom 30 indicated in FIG. 4. both inbound component 100-8 

defines a network common structure which is shared by the 80(1 outbound component 100-12 forward each packet to 

mechanism 100 and the so-called address resolution code output component 100-14. Component 100-14 includes rou- 

which can be viewed as standard. The if_name field is used tmc (HND— INPTJT_TYPE) which invokes a kernel ser- 

to define the virtual host system name (e.g. veO) while the routine for sending each such packet back to the local 

ve_flags field is used for storing a private flag. The state 55 nost network interface. 

field defines the state of the virtual host system while the The initialization component 100-4 includes a number of 
dient_count field defines the number of different client routines for performing the operations required for initial- 
processes in the table. The client table pointer field defines izing the virtual network mechanism 100 and the sets of 
the address of the first client table as indicated in FIG. 5. The virtual host control structures inet, ve_softc and client table 
local TP address field is used for staring a commonly used 60 control structures associated with each of the virtual host 
local host IP address value while the virtual host IP address systems ve0-ve3. 

field is used for storing a unique virtual host IP address 

value. By using a common local host IP address, this DESCRIPTION OF OPERATION 

elirninates the need to replicate the software routines of the With reference to FIGS. 1 through 8. the operation of the 

virtual network interface 100-2 of FIG. Z &3 preferred embodiment of the virtual network mechanism 

As indicated in FIG. 5. the client table data structure 100 of the present invention will now be described. By way 

includes the fields tcp_state through timer count as indi- of example, it is assumed that a number of client user 
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processes running on the remote host system 20 of FIG. 2 ing form: route add -net 213 65 43 123 45 ft? it™, 

want to utiLze the emulated system FTP services application vie "215.65.43- sr^« a^cuS^erlork ^r« 

prognun 22 running on host system 54. In accordance with argument (networkJ<£ess) whUefcTvaU^S 456 7- 

the teachings of the present invention, host system 54 is specifies a particular gateway address parameter (gateway 

configured to attach to the TP layer, a plurality of network 5 address). Once (he route add command is executed a 

interfaces, one for each emulating hosted operating system/ configures the static route for connecting to emulated system 

virtual host which are utilized by virtual network mechanism appUcation programs. As previously discussed, gateways 

100 to communicate with the IP layer. When so configured. 0811 statically or dynamically configured in a manner with 

the virtual network mechanism 100 operates with the dif- is w eU-known in the art 

ferent sets of structures, each of which has the local host IP io Additionally, the host system 54 must also configure the 
address and its own virtual host IP address. local host IP address for virtual network mechanism 100-2 
By way of example, it will be assumed that the IP address to communicate with the virtual host systems ve0-ve3. 
of the local host system has the value 215.65.43.1 wherein According to the present inven tion, this may be done by 
the value "215.65.43" designates the network address of the means of separate "VIRNET" directives included in the 
virtual LAN and the value"!" designates the address of the is hosted system (emulator) configuration file clm__x file. Each 
local host system connected to the virtual LAN. It will be VIRNET directive has the following format: VIRNET ve(n) 
appreciated that the values selected could have any numeri- fctl_args] wherein the first argument "ve(n)" specifies fee 
cal value as long as they are selected according to the particular virtual network interface mechanism 100 accord- 
standard internetwork conventions. That is, just as in any ^8 l ° "n" which has the values "0" through "3." 
network, each connection point or node must be assigned an 20 The remaining arguments include an address, up and 
IP address. Accordingly, each emulated system/ virtual host down arguments. The "address" argument corresponds to 
100-4a through 100-4rf running the TCP appUcation pro- cither a nost name or an IP address in the standard dotted 
gram which shown as connecting to the virtual LAN must decimal notation. The address used for this argument is 
also be assigned its own TP address. assigned to the host side of the virtual network interface 
By way of example, it is assumed that the virtual host 25 mechanism 100 (i.e.. local host interface 100-2). This 
100-4a has an IP address value of "2 15.65.43.2" wherein the address is automatically incremented by one to create the TP 
value "215.65.43" again designates the network address of address for *e first virtual host system veO connected to the 
the virtual LAN and the value "2" designates the virtual host vutulJ LAN on me opposite side of virtual network mecha- 
address of the emulated system/virtual host 100-4 which nism 100-2. The "up" argument is used to activate the virtual 
connects to the virtual LAN. Each of the other virtual host 30 networkinterfa ce mechanism 100-2 while the "down" argu- 
systems vel through ve3 has TP address values which ment is used to deactivate the virtual network interface 
corresponds to the incremented TP address of its local host mechanism 100-2. 

system (eg. vel, ve2, ve3) as for example, address values Whcn a VIRNHr directive is used in this example to 

"215.65.43.3" through "215.65.43.5." Again, the value configure the first virtual host l«Ma which connects the 
"215.65.43" designates the network address of the virtual 35 virtual network mechanism, the directive has the following 

LAN and the values "3" through "5" designate the virtual fona: VIRNEr ve « 215.65.43.1 up wherein the value 

host TP addresses of the virtual host systems 100-46 through "215.65.43.1" corresponds to the local host IP address and 

100-4J connected to the virtual LAN. It will be understood "215-65.43.2" corresponds to the virtual host TP and "up" 

that the TP virtual addresses and the network LAN could specifies the activation of the mechanism 100. The VIRNET 
have other values. <w directive is entered into the hosted operating system 

It wfll be appreciated that host system 54 which connects ( emuia f or ) clm_x file and is used for loading and config- 

to "reaT LAN 18 also has its own TP address which is " nag VUtUal network mechanism 100 software into the 

assumed to correspond to the value "192.45.6.7" while it is °P eratmg system kernel of host system 54. Other VIRNET 

assumed that the remote host system 20 has an TP address of ~ ecUves similar forms can be used to configure 

192.45.6.8. The value "192.45.6" corresponds to the net- 45 °? es of *** vir1ual hosts vel mrou * ve 3. In the 

work address while the host address values ~7" and "8" convent »on used by the present invention, it is not necessary 

designate host system 54 and remote host system 20 resoec '° again sctup the local host address since it was previ- 

tively. ' ously configured when the first virtual host veO was config- 

It can be seen that when so configured, system 54 can be ^ K tlvt^L"** fT** 0 * SyStem use * me 
viewed as actually being conue^^twT^eparanTaS 1^- !f 7^ aU ° WS fte USe of me «™ 
distinct LANs. Therefore, when^nl host^lJ sc^vare rouUnes included as par. of virtual network inter- 
wants to communicate with any application programs (e.g. Tf . . . . . 

FTP. TELNET) of emulated system/virtual hosts 100-fc, V. svstem 18 001 configured via directive, it 
through 10<Md which actually correspond to separate copies „ f^^^^^^atmg system comnvuidlme using 

of ES components running under the control ofthc operating tS^r ^ nlm f nd J*?* ««« same function as the 

system of host system 54. system 20 just has to configure the ^Zf^I^T^ ?"1 comman<l has me format: hvx_ 

local host system 54 to function as a "gateway" in die same vecf ? f e(n) [ctl_args] wherein "n" is used to designate the 

way it would configure a host system connected to a "reaT s P e ° fl = virtual host system (e.g. veO. vel. etc.). The argu- 

LAN. ments ctl_args are the same as those of the VTNBT direc- 



in t* system of the preferred embodiment, configuring is " 5^^352252 l^^SSE? 
detail, the route add command used to connect the vimiai Ac _ * " . ^ aooress ot ^15.65.43.1. 
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215.43.2 fa- the fast virtual host system veO running the ioctl command adds the JP address (eg. 215.65.43.1) to the 

emulating hosted operating system. The other virtual host arpcom control structure. This local IP address which is used 

systems are similarly configured but without performing any for mapping, is saved in the local IP address portion of the 

further increment operation. structure ve_softc of FIG. 5 as indicated in block 720. The 

. 3 system also computes the network and host portions for the 

Initialization virtual host system 100-4a. as indicated in block 722. 

The above described configuration operations can be In the preferred embodiment as discussed above, (he 

assumed to take place as part of the loading and start up of virtual host IP address for virtual host veO is generated by 

each emulator 80 of FIGS, la and lb which is to be run. adding one to the local host IP address (i.e.. 215.65.43.1). 

Such operations are represented by block 600 in the flow 10 The resulting value(i.e..215.65.43.2)forvirtualhost 100-4a 

diagram of FIG. 6. The load operation involves performing is saved in the virtual host IP address portion of the control 

the required configuration tasks, such as configuring the structure ve_softc of FIG. 5. Next as indicated in block 

different TCP/IP application programs (i.e.. servers) and 724. the host system sets the IFF_UP flag of the if_flags 

configuring the IP address for the associated virtual host field of the ifnet structure for the virtual network mechanism 

system using the VKNET directive included in the clm_x 15 100 to a state which indicates that the interface is "up." 

file. Additionally, the route command is used on the remote As seen from FIG. lb. a second type of ioctl command 

host to configure a gateway for the host system 54 to which (LcSIOCSIFFLAGS) is executed which sets the interface 

the remote host system is to be connected. This completes IFF_RUNNING flag to indicate that the interface is "run- 

the operations of block 600. ning." This enables the allocation of resources by the system 

Next, as shown in FIG. 6. the host system performs the which places the virtual network mechanism 100 in an 

. initialization operations of block 602. These operations are operative (running) state as indicated in block 730. The 

shown in greater detail in FIG. la. Referring to FIG. la, it above sequence of operations of FIGS, la and To is repeated 

is seen that host system 54 first obtains the unit number for each of the other configured virtual host systems 100-46 

value from the configuration file which the host system 54 through 100-44 

uses to locate the ve_softc control structure which defines 23 Referring to FIG. 6. once initialization has been 

the first virtual host system 100 -4a. Next, system 54 sets up completed, the virtual network mechanism 100 is ready to 

the various elements of the ve soft control structure 500 receive packets from remote system 20 specifying any one 

shownin FIG. 5 as indicated in block 700. That is. the of the virtual host systems ve0-ve3. As discussed above, the 

appropriate parameter values are loaded into the eight fields remote system 20 sends packets to the host having IP 

illustrated in FIG. 5. More specifically the fields are initial- address 215.6543.1 via the IP module of local host system 

ized as follows: the arpcom struct name to the "Ethernet 54 which operates as a "gateway." That is. the IP module 

common part", the ve_flags, the state of the interface to receives each data packet and determines that the data 

zero, the client_count value is set to zero (maximum packet should be routed to one of me virtual host systems 

value=512 which is an arbitrary value), the client table ve0-ve3 through the virtual network interface as specified 

. pointer value which specifies the location of the first client by the local host IP address, 

table structure is set to zero, and the local IP and virtual IP The IP module of host system 54 determines the IP 

addresses :are settozera Next, the host system initializes the address of the virtual host system (interface 100-2) from the 

chent table entry of FIG. 5 as indicated in block 702. More system network list. The IP module/layer then invokes/calls 

specifically, the fields tcp_state through timer count are ^ the virtual host output routine using the previously stored 

initialized to zeros. output© routine address (see block 704 of FIG. Id) contained 

Next, as indicated in block 704. the host system 54 builds in the ifnet structure associated with the designated virtual 

the ifnet structure of FIG. 7c for the virtual host system host system (e.g. veO. vel, ve2 or ve3). The IP module 

100-4a. It initializes its fields so that it contains with the includes in the call, all of the parameters required for 
addresses of the interface functions/routines (Le.. if__output, +J processing the included packet by mechanism 100. The call 

if_ioctl and if_rcset) utilized by the virtual network mecha- includes as a parameter, an address pointer to ifnet structure 

nism 100. Additionally, the appropriate value designating associated with the specific virtual host system. As indicated 

the type of interface which is "Ethernet" in the present . in block 6M, mechanism 100 accesses the ifnet structure to 

example, is also loaded into the structure. As indicated in obtain the unit number value designating the ve_softc 
block 705. system 54 saves the unit number value identify- y, control structure associated with the designated virtual host 

ing the virtual host system veO in the if_unit portion of the system. 

associated ifnet structure. As indicated in block 620 of FIG. 6. the mechanism 100 

Next, as indicated in block 706. the host system calls the processes the physical network (e.g. Ethernet) header in a 

if_attach kernel services of the ADC network interface standard manner. Next, as indicated 1 in block 608. the 
device software layer which adds the virtual network mecha- 55 mechanism 100 verifies the IP and TCP packets to ensure 

nism 100 as another network interface to the system wide that they have no errors. As indicated in Hock 610. the 

network interface list. That is. the configured ifnet and mechanism 100 next tests the protocol type value to deter- 

ve_softc structures are properly registered. Also, as indi- mine what type of network protocol is being used, 

cated in Mock 708, the host system turns on the timer As indicated above, it may be desirable to utilize multiple 
function which provides an arbitrary value (e.g. 20 minute) M virtual host systems to take advantage of multiple processor 

time interval to dean out stale client table entries. This resources of a multiprocessor system. In such cases, it is 

completes this portion of the initialization sequence of block only necessary to provide one type of protocol, such as a 

m ~ specific Ethernet protocol. In other instances, multiple vir- 

Next, as part of the initialization sequence, the host tual host systems may be used to operate in conjunction with 
system executes an ioctl command (ie.. SIOCS1FADDR) as 63 Different types of physical networks, such as Ethernet Token 

indicated in FIG. 76. This command is used to set the Ring. FDDI. etc or operate in conjunction with different 

network interface address. As indicated in block 720. the protocols of a specific type of physical network, such as 
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Ethernet From an implementation point of view, it may be 
desirable to utilize a separate virtual LAN for each different 
physical network media (e.g. Ethernet, Token Ring. FDDI). 
In this instance, it is necessary to replicate virtual network 
interface 100-2 within each virtual LAN and assign each 5 
such network interface, a different local host IP address 
value. 

For ease of explanation, it is assumed that each of the 
virtual host systems 100-4o through 100-4J provide differ- 
ent types of Ethernet protocols. If it is a specific one of the i° 
types of Ethernet protocol (i.e- has a hexadecimal value of 
800), then the mechanism 100 next checks for the type of IP 
protocol by examining a type field contained in the IP 
packet If it is not a specific Ethernet protocol, then the 
mechanism 100 drops the packet as indicated in block 612. is 

As indicated in blocks 616 and 618. when the IP protocol 
type field specifies ICMP. the mechanism 100 performs echo 
processing wherein it echoes the packet and then calls the 
kernel services function find_Jnput_typeO. This function 
automatically deposits the packet into the IP module. When 20 
the IP protocol type field specifies TCP. then the mechanism 
100 determines if the packet originated from a local or 
remote host system as indicated by block 620. When the 
packet originates from a local host mechanism 100 invokes 
the outbound function as indicated in block 622. When the 25 
packet originates from a remote host mechanism 100 
invokes the inbound function as indicated in block 630. 

The inbound function is shown in greater detail in FIG. 
Id. As indicated in block 752 of FIG. Id. mechanism 100 
searches the set of virtual host client table(s) for this packet 30 
As discussed, this involves searching up to 512 client tables 
to make certain that the client/user exists (i.e.. a client table 
was opened/allocated for that particular client). If mecha- 
nism 100 determines that the client does not exist (per block 
752). then mechanism 100 allocates a table entry for the " 
client as indicated in block 754. More specifically, mecha- 
nism 100 establishes a client table entry for that client such 
as shown in FIG. 5 and increments (he client_count field by 
one. As indicated in block 756, the mechanism 100 saves the 
32-bit client source IP address (ip_src), the 32 bit destirta- *° 
tion IP address (ip_dst) and 16 bit TCP source port (th_ 
sport) and destination port (th_dport) numbers such as 
indicated in FIG. 7c. 

Next, as indicated in block 758, nwhanjTm im over- A5 
writes the destination IP address (ip_dst) with the value 
obtained from the local host IP address field previously 
stored in the control structure ve_softc of FIG. 5. Now. the 
packet identifies the local host as the destination so that the 
packet wfll be processed by the host IP module. Mechanism 50 
100 then overwrites the source IP address (ip _jsrc) with the 
uniquely assigned value obtained from the virtual host IP 
address field of control structure ve_softc associated with 
the particular virtual host system as indicated in block 760. 
This now identifies mechanism 100 as the source of the 55 
packet so that any response by the ES FTP services appli- 
cation server will be returned back to mechanism 100/virtual 
host system for rerouting back to the original source, remote 
system 20. The mechanism 100 next recalculates a new IP 
checksum word (tp_cksum) which is overwritten into the IP M 
packet header checksum field of FIG. 7e as indicated in 
block 762. 

Next, mechanism 100 overwrites the "well-known" TCP 
destination peat number (th_dport) with the mapped port 
number value as indicated in block 764. The mapped port 65 
number value is a port number which identifies the ES FTP 
application server 22 of FIG. 2. The mechanism 100 maps 
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the well-known port number into a non-well-known port 
number value. The mapping is carried out in a relatively 
simple matter for example, the well-known port number 
value "21" is changed to "5021." It wfll be appreciated that 
the ES FTP application server 22 will have been previously 
configured to listen on port "5021" instead of the well- 
known port "21". This is done by entering the value "5021" 
into the appropriate services file. It wfll be noted that each 
of the other virtual host systems wfll have a unique mapped 
value. For example, the mapped values for virtual host 
systems vel, ve2 and ve3 for the well-known port number 
value "21" may correspond to "6021." '7021" and "8021." 
respectively. It wfll be appreciated that any value could have 
been used as the mapped value. For tracking purposes, it is 
advantageous to select a value which also contains the 
well-known port number value. This simplifies and speeds 
up the mapping process which can be implemented as a 
masking operation, eliminating the need to account for 
carries, borrows, etc. 

Next as indicated in block 766. mechanism 100 maps the 
index value obtained from the client table pointer field of the 
particular virtual host control structure ve_softc as the TCP 
source port number (th_sport). The index value (e.g. ZERO 
initially) is used to overwrite the th_sport field of the TCP 
header of the packet as indicated in FIG. 7e. This virtual port 
number is used as a temporary port number which provides 
an index associated with the particular client/user table. 
Mechanism 100 is able to use the virtual source port number 
as an index into the client/user tables. This index number 
arrangement facilitates packet processing by reducing the 
amount of search time in locating the appropriate client 
information for the reply packet. 

Mechanism 100 then calculates a new TCP checksum as 
indicated in block 768 and uses the sum to overwrite the 
th_sum portion of the packet TCP header as indicated in 
FIG. 7e. Next mechanism 100 sets the top state filed to an 
appropriate state in the client table structure which enables 
mechanism 100 release the client table entry. Also, mecha- 
nism 100 resets the timer count word to zero as indicated in 
block 770. Following the completion of the operations of 
block 770. mechanism 100 calls the kernel services find_ 
input— typeO function. The call includes all of the parameters 
required for sending the modified packet to the host system 
IP layer/module. 

It will be noted that the only portions of the inbound 
packet which are modified are the source IP address and 
destination IP address as well as the TCP source and 
destination port number values. The remaining portion of the 
packet are maintained as the same. Mechanism 100 recal- 
culates the checksums to reflect these modifications and 
stores the new checksum values to the TCP and IP headers 
of the packet Because of the minimal changes made, 
mechanism 100 is able to carry out these operations within 
a minimum amount of time. 

The host IP module upon receiving (he mapped packet 
from mechanism 100 determines from the source IP local 
address that the packet is fox host system 54. The IP module 
processes the packet and send it to the TCP layer which 
forwards the packet to the EX FTP application server 22 as 
designated by virtual destination port number (th_dport) 
which corresponds to the value "5021" in the example. 

After the ES FTP application server 22 processes the 
packet it normally generates a response/reply packet in a 
conventional manner. This packet is also formatted as shown 
in FIG. 7g which is the same as the format of FIG. 7e. Here, 
the server 22 includes the same virtual source and destina-' 
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tion port numbers in the packet's TCP header in addition to also resets to zero, the timer count word contained in the 

including the same source TP and destination IP addresses. client table structure. As indicated in block 798 mechanism 

Since the server 22 is the source of the response packet, the 10© calls the kernel services function find_jnput_typeO 

sets of values are reversed to indicate server 22 as the source which sends the response packet to the local host IP module 

or sender of the response packet and mechanism 160 as the s The IP module based upon the IP address automatically 

destination or recipient of the response packet routes the response packet to the remote host system 20 via 

The host TCP/IP stack passes the response packet through 'ocal area network 18. 
both the TCP and IP layers/modules for processing in a Subsequent packets sent by the client application program 
conventional manner which results in the packet being of remote host system 20 are automatically routed to the 
forwarded to the designated virtual hostfmechanism 100 in >° particular virtual host system/mechanism 100 which pro- 
accordance with the specified packet virtual IP destination cesses each packet through the inbound function in the 
address - manner indicated in FIG. Id. Since mechanism 100 previ- 

As indicated in FIG. 6, the IP module passes the packet to °usly allocated a table entry to the remote system client 

the designated virtual host system by invoking the outputO application program, the operation of block 754 is omitted, 

function in the same manner described above. Briefly, the IP 15 Similarly, any packets returned by the ES FTP application 

module passes all required arguments/parameters including server 22 are processed by mechanism 100 through the 

the specified ifnet structure pointer. Mechanism 100 again outbound function in the manner indicated in FIG. 7g. 

performs the operations of blocks 606 through 620. When If for any reason, the client application program fails to 

mechanism 100 checks the originator of the packet, as send packets for a long period of time because of a line 
indicated in block 620, it determines that the response packet 20 disconnect or similar condition, mechanism 100 allows the 

is from local host system 54. This causes mechanism 100 to continued incrementing of the timer count word without 

invoke the outbound function of block 622. This function is resetting same. Therefore, when mechanism 100 initiates a 

shown in greater detail in FIG. 7/ scan of the virtual host system's client table structures, it 

Referring to FIG. If. as indicated in block 780, mecha- detects that the timer count word of the client table structure 

nism 100 converts the virtual TCP destination port number associated with the client. application program will have 

(th_dpart) assumed initially to have the value of zero, into exceeded a predetermined count indicating lack of activity, 

the client table slot entry. It uses this value as an index to In such instances, mechanism 100 deallocates or clears the 

obtain the previously saved client information (i.e., stared in client table structure entry thereby freeing up space and 

the allocated client table structure of FIG. Sb) as indicated eliminating stale entries. 

in block 782. In this example, the zero index value is used 30 FIG. 8 illustrates diagrammaticalry. the overall operation 

£J°!ff thc 5 SOCUit ! d , clicnt **** structure. Mechanism of the mechanism of the present invention. As shown. 

■ taW r< ^ CVCS 01 addreSS 54011x1 °* dient re °«>te host system 20 initiates a connection with ES FTP 

table structure. application server 22 through a connection packet which is 
As indicated in Mock 784. mechanism 100 overwrites the 3S indicated by the path labeled "1." Next, mechanism 100 

destination IP address (ip_dst) of the packet IP header with maps the connection packet and routes the packet to the 

the saved source IP address identifying the remote host server 22 as indicated by the path labeled "2." Any response 

system 20 as the destination for the packet Next as indi- packets from server 22 are sent to mechanism 100 as 

cated in block 786, mechanism 100 overwrites the source IP indicated by the path labeled "3." Mechanism 100 remaps 
address (ip_src) with the saved virtual host IP address w each such response packet and sends it to the remote host 

identifying virtual host system/network mechanism 100 as system 20 as indicated by the path labeled 4 

rf . mc 50 subsequent packets will be From FIG. 8 and the above descriptions, it is seen how the 

^ntTf^fTT ^^Tt m W0 ^ 78S - ™**anisn> of the present inventions host and hosted 

* ^ * 3 f"* ch «* sum word <** system application programs executable by multiple emu- 
ZTT^L? ^ST mt0thcIP £ e f dcr checksum I**" 45 lating hosted operating systems/virtual host systems sharing 

faon (,p_cksum> of the response packet a single host TCP/IP communications network stack to use 

Mechanism 100 then retrieves the saved client TCP the same well-known port without having to make any 

source (sre) port and destination (dst) port numbers from the changes in dient application programs. The mechanism of 

client table structure of FIG. 5. As indicated in block 790. the present invention by operating below the IP layer of a 
mechanism overwrites the TCP destination port number » network stack is able to take advantage of the routing 

information (th_dport) contained in the response packet's capabilities of the IP layer/module. This minimizes the 

TCP header with the previously saved client source port amount of software required to be added to the host oper- 

number. This change now identifies the remote host system ating system facilities in incorporating the virtual network 

20 TCP layer as the destination for the response packet mechanism of the present invention. 

™~ ™° < * m !? anisin °VHwrites 55 Those skilled in the art will appreciate that many changes 

E^^f * source Port nu^ information may be made to the r^erred^nbodimcnt tfttepS 

(th_spart) contained in the packet s TCP header with the invention without departing form its teachings ForexS 

^^I^^XZTr^^t ^ M 1-vious.y descS^fprcscnt in^eS can be uS 

F.f chan 8 e - * c response packet now with different types of communication network protocols 

t^S^i^ST^^ 100 as the go such as Emen^oken-Ring. FDDL etc JEE^S 

source of the response packet invention could also utilize other types of mapping tech- 

Again, as indicated in block 794, mechanism 100 calcu- niques to generate the required virtual identifier information 

lates a new TCP header checksum word which is used to utilized in conjunction with the forwarding of packets 

overwrite the TCP checksum (th_j>um) value contained in through the TCP/IP network protocol stack. Other modifi- 

the response packet TCP header as indicated in FIG. 7g. 65 cations of this type relative to protocols, data structure 

Mechanism 100 adjusts the tcp_state value contained in the formats, operating system facilities/calls and the like will 

client table structure of FIG. 56 as indicated in block 796. B also occur to those skilled in the art. Further, the present 
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anTn. • ™T ^1 ^ tost stan °n address identifier and well-known service 

We in accordance with the provisions and statutes there identifier values so each outgoes mhS slnt^ 

^» and the best form of the S the virtual network mechS Ke^S 

mvenhon certain changes may be made without departing appears to the remote host system Is a rcpTy^ctaUo 

from the spirit of the invention as set forth in the appended the communication between tetn£^J?ZZ£~. 

claims and that in some cases, certain features* the andAehosted^m aS^n ^sif we^ 

vLfTc^t- 10 ^ address assi 6 n «» <° »e paTticuIar hosted 

i a J!*I?T w „ , . operating system with the well-known services identi- 

1. A method which allows a local host system to share a fier value, 

network software facility of the local host system operating 2. The method of claim 1 wherein the virtual network 

system between a number of application servers operating mechanism includes interfacing software similar to the 

under the host operating system and a corresponding number network interface unit and a common set of software rou- 

of application servers operating under components of a tines utilized by each of the plurality of virtual host systems 

plurality of hosted operating systems running under control 3 method of claim 2 wherein the network software 

of the local host operating system, the local host system facility includes a TCP/TP protocol stack containing TCP and 

being coupled to at least one remote host system through a ff layers 80,1 ^ virtual network mechanism utilizes the 

local area network (LAN) and an internetwork, the network network routing capabilities of the IP layer, 
software facility being coupled to a network interface unit 20 4 ' • The metnod of dain » 1 wherein the standard commu- 

which includes interfacing hardware and software for con- ""cations network protocol is the TCP/IP protocol, the 

necting the local host system to the LAN for communicatina • *** ldcntMcr valuc corresponds to an IP address 

with the remote host system using a standard communica- con |^ mn S W source and IP destination addresses and the 

tions network protocol which is Aarartrriral by assigning wc "j? own scrvicc function identifier value corresponds to 

different station address identifier values to each host system M * weU " known P°rt number value containing TCP 

and well-known services function identifier values to the S °^L* nd "^P" dcstitmtion P™* numbers, 

different data communications application servers asscci- , ^ method of claim 1 wherein configuring step (a) of 

ated with local host system and hosted operating systems so mc J^ 00 "**»des the step of: 

that servers performing the same service function are (d) loading and initializing each of the plurality of hosted 

assigned the same well-known services function identifier 30 operating systems using a number of directives, 

value for directing incoming packets sent by the remote host 6 The mcthod of claim 1 wherein each different set of 

system to the appropriate application server, said method structurcs includes predetermined types of control data 

comprising the steps of: structures including a first structure which defines the exist- 

(a) configuring a virtual network mechanism within the ^ ° f * e P articular virtual host system to the network 
local host operating system to be cperatively coupled to " * bahty a second structure which defines the 
the host operating system network sofrwarefacUiry ^1?°* 2?T . 

through a plurality of network interface structures to • . . method of claim 6 wherein the first structure 

function as a virtual LAN connected to a plurality of 'tUT? a P lund,t y of Adds, a first field containing a name 

virtual host systems running the hosted operating sys- 7 - - me virtual host system and a second field 
tern with each virtual host system operating as if it 40 dca S nann 6 the sccond structure associated with the virtual 

contained its own network software facility; , 2^™"^ . 

(b) reallocating memory and initializing a different set of • Zi of claan * whercin * e structure is an 
structures in reallocated memory for each of the mterface network structure utilized by the host operating 
plurality of virtual host systems which operate in « S ^° nd sbuaan is a software control struc- 
conjunction with the virtual network mechanLi and " ^ 10 1Mnagc Passing for each of 
the plurality of hosted operating systems each different m = cuent application programs running on the remote host 
set of structures containing a unique unit number ,7 , a ^ X3 , S f 8 appljcatlon services running on that par- 
identifying me virtual host ^sterns^LZed ^ ^tr^rnl^^^^ 

with and a unique IP address designating the particular «, of ™^J^T^f 1 * e P rettet amined types 
virtual host system within the virtual LAN- 50 0 £, contro1 **** structures includes a number of client table 

(c) mapping predetermined portions of each incomine associated with 
packet by the virtual nerwTmecr^m sem byT sv^w^hH aM f^ pr0gran ° f me ren "* e h °* 
remote host system and received from the^host tiSvSaf .S SUf" » ^ 
communications network software faeffityrjyefaanrine ss 10 Th- m-i^ » v • 

the station address identifier value of each iriccS »JLJ^T ^ ^° a new cUent ,able * 

packet to specify the local host system a?Tde^3 ^ V , partJcular host system each time a 

Ld the parSr sourTof tL ^c^f "™ by * f*™ 1 

packet for returning any reply jacket and chZJog it *T^^ D £* SyStenL 

weU-known services ioentifier value to a^rZl host «, SV 3L^ Clmm 10 wherdn *e remote host 

identifier value so that me packe^ec^e^rom T H ^ abLsheS Conn f« lon ^ *e hosted operating 

virtual network mechanism Tdirect^Se network I^^^^Tf^ ° * a PP"cation servers of 

softwarefacmtytomear^opr^aS^s^cS r ^ m «^,f h ^ ^ by the 

mg reply packet sent by a hosted system application the virtual host systems. * 
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12. The method of claim 10 wherein the client table of 
each set of structures includes a predetermined number of 
fields, a first field for storing the station address identifier 
value of the remote system client application program, a 
second field defining the operational state of the client table, 
third and fourth fields for defining different client applica- 
tion program port identifier values and a fitch field for 
storing a ri mer count value defining client application pro- 
gram activity. 

13. The method of claim 8 wherein the second structure 
contains a predetermined number of fields, a first field 
designating the name of the virtual host system, a second 
field for storing the state of the virtual host system, a third 
field for maintaining a count of the number of different client 
entries being managed by the virtual network mechanism, 
fourth and fifth fields far storing the common local host and 
unique virtual host station address identifier values respec- 
tively and a sixth field for storing a client pointer value for 
accessing the first client table structure generated by the 
virtual host system. 

14. The method of claim 13 wherein the virtual host 
station value for a first one of the virtual host systems is 
generated by performing an arithmetic operation on the 
common local host station address identifier value. 

15. The method of claim 1 wherein each virtual host 
system is used to process packets transmitted utilizing one of 
a number of protocols defining a predetermined type of 
standard protocol. 

16. The method of claim 1 wherein the method further 
includes the step of: 

(f) saving the station address identifier value of the remote 
host system and the well-known services identifier 
value contained in each incoming packet in a client 
table structure generated by the particular virtual host 
system which can be indexed through the virtual iden- 
tifier in response to having received an initial connec- 
tion packet from a client application program running 
on the remote host system for enabling the subsequent 
mapping of each reply packet 

17. The method of claim 1 wherein the mapping step (a) 
of the method includes the step of mapping the weO-known 
services identifier value to a non-well-known services iden- 
tifier value containing the well-known services identifier 40 
value. 

18. A virtual network mechanism which allows a local 
host system to share a network software facility of the local 
host system operating system between a number of data 
communications application servers operating under the 43 
host operating system and a corresponding number of appli- 
cation servers operating under components of a plurality of 
hosted operating systems running under control of the local 
host operating system, the local host system being coupled 

to at least one remote host system through a local area so 
network (LAIN) and an internetwork, the network software 
facility being coupled to a network interface unit which 
includes interfacing hardware and software for connecting 
the local host system to the LAN for communicating with 
the remote host system using a standard communications 
network protocol which is characterized by assigning dif- 
ferent station address identifier values to each host system 
such that the local host system and hosted operating systems 
are assigned different station addresses and well-known 
services function identifier values to the different data com- 
munications application servers associated with local host 
system and each of the plurality of hosted operating systems 
so that servers performing the same service function are 
assigned the same well-known services function identifier 
value for directing incoming communication data packets 
sent by the remote host system to the ap p ropri ate commu- 
nications application server running on the particular hasted 
operating system, said mechanism comprising: 
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(a) an interface component configured within the local 
host operating system to operatively couple the virtual 
network mechanism to (he host op er ati ng system com- 
munications network software facility as a virtual LAN 
connected to a plurality of virtual host systems which 
are the components of the plurality of hosted operating 
systems; 

(b) an initialization component for preallocating and 
initializing a different set of structures for each of the 
plurality of virtual host systems which operate in 
conjunction with the virtual network mechanism and 
the plurality of hosted operating systems, each different 
set of structures being initialized to contain a unique 
number value identifying a particular one of the virtual 
host systems and a unique TP address designating the 
virtual host system on the virtual LAN; 

(c) a first mapping component coupled to the interface 
component far mapping predetermined portions of 
each incoming packet sent by the remote host system 
and received from the interface component through the 
local host network software facility so that the station 
address identifier value of each incoming packet is 
changed to specify the common local host system as a 
destination and the particular virtual host system as a 
source of the packet for processing each reply packet 
and the well-known services identifier value is changed 
to a virtual identifier value so that the packet received 
from the virtual network mechanism is directed by the 
network software facility to the appropriate application 
server of the designated hosted operating system for 
processing; and, 

(d) a second mapping component for mapping the prede- 
termined portions of each outgoing reply packet sent by 
a hosted system communications application server 
through the network software facility to the interface 
component by restoring the remote host station address 
identifier and well-known service identifier values so 
each outgoing reply packet sent by the virtual network 
mec hanism to the internetwork appears to the remote 
host system as a reply packet to the communication 
initiated by a client application program running on the 
remote host system and the hosted system application 
server as if the server had been accessed through the 
LAN using the originally sent station address assigned 
to the particular hosted operating system by the well- 
known services identifier value. 

19. The mechanism of claim 18 wherein each set of 
control structures includes a first structure which defines the 
existence of the virtual host system to the network software 
facility and a second structure which defines the virtual host 
system operational status. 

20. The mechanism of daim 19 wherein the first structure 
is an interface network structure utilized by the host oper- 
ating system to communicate with the virtual host system 
network facility and the second structure is a software 
control structure which the virtual host system uses to 
manage packet processing for each of the client application 
programs running on the remote host system- the software 
control structure containing a predetermined number of 
fields, a first field designating the name of the virtual host 
system, a second field for storing the state of the virtual host 
system, a third field for maintaining a count of the number 
of different client entries being managed by the virtual 
network mechanism, fourth and fifth fields for storing the 
common local host and unique virtual host station address 
identifier values respectively and a sixth field for storing a 
diem pointer value for accessing the first client table struc- 
ture generated by the virtual host system. 
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ABSTRACT 



A new type of data transport service which uses a frame 
relay layer 2 data link connection identifier (DLCI) to select 
among various service types, feature sets, and/or closed user 
groups (CUGs). A layer 3 address may be extracted from a 
layer 2 frame, and the layer 3 address information may be 
used to route a data packet over a packet-switched network 
according to the service classes, feature sets, and/or CUGs 
selected. At the destination, the layer 3 data packet may 
again be enclosed in a layer 2 frame with a DLCI indicating 
the service classes, features sets, and/or CUGs. Because the 
use of conventional permanent virtual circuits (PVCs) is not 
required in aspects of the invention, new methods of mea- 
suring and managing network traffic are presented. 

10 Claims, 10 Drawing Sheets 
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1 2 

FRAME RELAY SWITCHED DATA SERVICE At the UNI 920, the frame is checked for validity to 

determine if there is a predefined PVC associated with the 
The present application claims priority from copending DLCI 912. If so, the frame 914 is then forwarded on that 
provisional application Ser. No. 60/051,564 entitled pvc through the network along the same path and in the 
"FRAME RELAY SWITCHED DATA SERVICE" filed on 5 same order as other frames with that DLCI, as depicted in 
Jul. 3, 1997, herein incorporated by reference, and is related 2 - Th e laver 2 frame information remains as the packet 

by subject matter to concurrently filed U.S. patent applica- traverses the frame relay network whether this network is 
tion Ser. No. (attorney docket no. 03493.20133), entitled actually implemented as a frame relay network or other 
"TRAFFIC MANAGEMENT FOR FRAME RELAY network such as an ATM network. The frame is carried to its 
SWITCHED DATA SERVICE" by the same inventors 10 dcstination without any further routing decisions being made 

in the network. The FCS is checked at the egress UNI, and 
BACKGROUND OF THE INVENTION tf me &ame is not corrupted, it is then output to the UNI 

1. Technical Field associated with the end user. 

t, ,. ... , . As is well known in the art, FIGS. 1-3 provide exemplary 

The present invention is directed to systems and methods diagrams of how me frame relay data packets are assembled 
lor implementing improved network architectures, and more 15 at the various ISO layers using the example of TCP/IP 
specifically to systems and methods for routing internet protocol transport over a frame relay data link layer. The 
protocol (IP) packets using modified frame relay protocols. example shows how the user data at the application layer is 

2. Description of the Related Arts "wrapped" in succeeding envelopes, making up the PDUs, 
Recently, the popularity of large "meshed" networks has aS * P 355 ^ down protocol stack. Specifically, the corn- 
been increasing. However, large-scale highly-meshed net- M P"* 511 "" 1 of ,ne Header field is expanded for detail and is 
works can be difficult to implement, maintain, and manage « ? 7° FIG ' 5 ; J?. e data ^ connect i°n identifier (DLCI) 

using conventional network technologies i d ^? mp " seS ^ ltS Spread over me firsl 40(1 second octet > 

A „ . e and allows for 1023 possible addresses, of which some are 

An example of a conventional mesh configuration is reserved for specific uses by the standards As shown in FIG 
shown in FIG. 1. A wide-area network (WAN) 900 includes * 3, the DLCI £ added to theframe relay heliSK £ 
a plurality ot routers R,,, R B , R c , R e , (customer premises what destination IP address is specified in the IPpacket This 
equipment (CPE)) respectively disposed at a plurality of end decision about what DLCI is chosen is made by the CPE, 
user locations A, B, C, and D and interconnected to a service usually a router, based on configuration information pro- 
provider's network (SPN) 901 via respective user-network vided by the customer that provides a mapping of IP 
interfaces (UNI) 920-1, -2, . . . , -n. The user-network 30 addresses into the PVCs that connect the current location 
interfaces 920 may be variously configured to be, for with others across the WAN 900. 

example, an asynchronous transfer mode (AIM switch hav- In conventional flame relay, a layer 2 Q 922 frame carries ■ 

ing a frame relay interface to CPE. Connecting the sites the layer 3 customer data packet across the network in a 

together are logical paths called, for example, permanent permanent virtual circuit (PVC) which is identified by a data 
virtual circuits (PVCs) P A ^, P^, P g _ D , p A _ B , ? CJ>y th at m 3J unk connection identifier (DLCI). Thus, the DLCIs are used 

characterized by their endpoints at the UNIs 920-1, by the customer as addresses that select the proper PVC to 

VZU-2,, . , 920-n and a guaranteed bandwidth called the carry the data to the desired destination. The customer data 

committed information rate (CIR). packet is carried across the network transparently and its 

FIG. 2 provides a detailed view of the flow of data across contents is never examined by the network, 
the WAN 900. There exists a plurality of layers of protocol 40 The conventional meshed frame relay network discussed 

over which communications may occur. For example, the above has a number of limitations. For example every time 

well-known layers of the International Standards Organiza- a new end user location is added to the meshed' network, a 

lion's (ISO) Open Systems Interconnect Model having lay- new connection is required to be added to every other end 

ers from a physical layer (layer 1), a datalink layer (layer 2), user location. Consequently, all of the routing tables must be 

a network layer (layer 4), up through and including an 45 updated at every end user location. Thus a "ripple" effect 

application layer (layer 7). Under this model, user data 902 propagates across the entire network whenever there is a 

is generated by a user application running at the application change in the network topology. For large networks with 

layer 903. At the transport layer (layer 4) 904, a source and thousands of end user locations, this ripple effect creates a 

destination port address 906 (as part of the TCP header large burden on both the network provider to supply enough 

(layer 4)) may be added to the user data 902. At the network 50 permanent virtual circuits (PVCs) and on the network cus- 

layer (layer 3) 905, an additional header (i.e., an IP header tomers in updating all of their routing tables Further most 

(layer 3)) containing source and destination IP addresses) routers are limited to peering with a maximum of 10 other 

908 may be added. Thus, the layer 3 user data field includes routers which makes this network topology difficult to 

the layer 4 user data 902 plus the layer 4 header 906. The implement. As networks grow in size, the number of PVCs 

layer 3 protocol data unit (PDI) 902, 906, 908, which makes 55 customers need to manage and map to DLCIs increases 

up, for example, an IP packet 950, is then passed down to Further complicating the problem is a trend toward increas- 

layer 2 909 in the CPE (routers R^, R^, R c , R D ) that ing "mesbedness" of networks, meaning more sites are 

interfaces to the SPN 901. In the router, a table maps one or directly connected to each other. The result is a growth in the 

more IP addresses (layer 3) 908 to an appropriate PVC or number and mesh of PVCs in networks that does not scale 

PVCs (P^^-, P A . D , P B . D , P A ^, Peg) The router table is 60 well with current network technologies 

maintained by the customer. Once the correct PVC is located Apossible solution for handling large meshed networks is 

Me^fT™ ™ n T£?° ndin f hDS t connection <° use a virtual private network (VPN) which interconnects 
identifier (DLCI) (layer 2) 912 u> coded into the header of end user locations using encrypted traffic sent via "tunnel- 
foe frame relay frame 914 (packet). Thereafter, the remain- ing" over the internet However, VPNs are not widely 
l"°™ e reh J r ^ * mcluded Md a frame check 65 supported by internet service providers (ISPs), have erratic 
sum (FCS) is computed. The frame is then passed down to information rates, and present a number of security con- 
the physical layer and transmitted to the SPN 901. cerns. 
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Another passible solution is the use of frame relay based customers because, unlike that of conventional frame relay 

switched virtual crcuits (SVCs). While PVCs (discussed customers no longer need to update their local DLCI tables 

above) are usually defined on a subscription basis and are each time a network customer with whom they wish to 

analogous to leased lines, SVCs are temporary, defined on an communicate is added or removed from the network Thus 

as-needed basis, and are analogous to telephone calls, s the customer's burden of network administration is substan- 

However, SVCs require continuous communications daily reduced. 

between all routers in the system to coordinate the SVCs i„ , , c .x. ■ .■ ~. ™ 

Further, because the tables mapping IP addresses to SVC tn * ^"P"* of "™ ntl °°> DLCfa may be used 

address.saretypicaflymanuau/Lmtamed.S^arLf.en g, r,S w hT g .T"* "TT CategWy 

impractical for large highly-meshed networks. Security is a to ^ to se lee t Sn^nTfv?? ^ "Sir 

major concern for SVC networks where tables are misman- ? Wentional DF CIs"W > L 1 , .1 

aged or the network is spoofed. Further frame SVCs are ( . convcDb ° nal P L{ ~ k } } D other w °rds, conventional frame 

difficult to interwork with asynchronous u^lT mode ^Te tnT* J"? T * ° f ^ l™? 1 ™™*™ 
(ATM) SVCs within the same network, allowing aspects of the present 

. T ... ' , . , invention to be incrementally implemented in existing con- 
None of the above solutions adequately address the grow- is ventional frame relay networks, 
ing demand for large mesh networks. Accordingly, there is T _ f, lrl L„, „„„„ , e ■ ,. ... 
a need for network architectures which enable implmen\a m "J r ° f Tf ^V^^ T™* 
tion of large mesh networks having security low mainte- 2Z%? , * h "n" ^ defined J b J r * e °P en 
nance cos.!, efficient operations, and scSty *Z ZTT? ° T com P are I d r wlth each otber in a 
v , ouu ^lauiuiy. ^ network to determine routing errors. If the addressing in the 

SUMMARY OF THE INVENTION ^ yefS consistent each other > tnen 1« associated data 

is routed without interruption. On the other hand, if the 

Aspects of the present invention solve one or more of the addressing in the layers is inconsistent with each other, the 
above-stated problems and/or provide improved systems associated data may be specially handled. For example, the 
and methods for implementing a network architecture. ^ data may be discarded, sent to a pre-determined address, 

A new type of data transport service takes advantage of and/or returned to the sender. This address comparison may 
the existing base of frame relay customer premises equip- be a PP ued to the sending address and/or the destination 
ment (CPE) and customers while offering a new mechanism address. An advantage of this multiple layer address com- 
for providing extensible service features to those customers. parison is that network security is increased. For instance, 
In the new service, data link connection identifiers (DLCIs) 30 P roblems suc h as "spoofing," which is the practice of ' 
may be used by the CPE to select among service types, Purposely providing an incorrect sending internet protocol 
feature sets, and closed user groups (CUGs). The DLCI is address, are better controlled by such a method, 

used in the layer 2 frame that conveys the user data to the ' n st *1 further aspects of the invention, routing look-up 

network. The layer 3 user data packet is extracted from the tables within the network are separated such that, for 
layer 2 frame and the layer 3 address information for the 35 example, each customer, closed user group (CUG), extranet, 
(mutable) protocol is used to route the user data packet over and/or intranet may have its own private partition and/or 
a high-performance packet switched network, according to separate table. This can provide greater network speed 
• the service class/feature set selected by the DLCI. At the because a router need not scan the entire available address 
destination, the layer 3 data packet is again enclosed in a space for all network customers at once. Furthermore, data 
layer 2 frame with a DLCI that indicates to which service 40 secur i'y is improved because the risk of sending data to a 
group it belongs. The frame is then forwarded to the CPE. wrong recipient is reduced. 

Use of this technique will allow the existing frame relay In yet further aspects of the invention, layer 3 and/or layer 
CPE to support, over the same physical interface, conven- 4 IP address information is utilized to route the fast packets 
tional frame relay service with a range of DLCIs that are through the network. 

1 ° git ;f 1 PathS SUCb P crmancnt viral ri^t 45 In even further aspects of the invention, new network 
(PVCs), as well as a range of DLCIs that are linked to traffic management techniques and measurements are 
service and/or feature sets. This will allow a robust method defined. For example, in some traffic-management aspects of 
for extension of new services to the frame relay installed the invention, committed delivery rates (CDRs) may be 
base, with minimal impact to existing customer equipment. assigned to one or more UNIs. A CDR is the average 
In some aspects of the invention, frame relay DLCIs are 50 minimum data rate that is guaranteed to be delivered to a 
used for selecting among various "service categories." This given UNI when sufficient traffic is being sent to the UNI. In 
differs significantly from conventional frame relay, which further traffic-management aspects of the invention, a des- 
uses DLCIs only to select PVCs and/or switched virtual filiation rate share (DRS) is assigned to one or more UNIs. 
circuits (SVCs). Service categories may include, but are not The DRS may be used to determine the share of traffic that 
limited to, communication via the public internet, commu- 55 a given UNI may send through the network. If several UNIs 
nication via a local intranet, communication within a closed are simultaneously offering to send traffic to the same 
user group (CUG), communication with an extranet (e.g., a destination UN, then each sending UNl's share of the 
network of trusted suppliers or corporate trading partners), network may be determined by its own DRS and the DRSs 
live audio/video transmission, multicasting, telephony over of the other sending UNIs. 

internet protocol (IP), or any combination thereof Thus, the 60 These and other features of the invention will be apparent 
concept of a frame relay PVC is significantly expanded by upon consideration of the following detailed description of 
aspects of the present invention. For example, the location of preferred embodiments. Although the invention has been 
an intended network endpomt recipient is not necessarily defined using the appended claims, these claims are exem- 
determmed by a DLCI at a sending network endpoint. The plary in that the invention is intended to include the elements 
DLCI may represent a service category with the intended 65 and steps described herein in any combination or subcom- 
recroient indicated by an IP address within the frame relay bination. Accordingly, there are any number of alternative 
packet. This results in a significant benefit to network combinations for defining the invention, which incorporate 
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one or more elements from the specification, including the 502, UNIs 402, and/or routers/switches 501 may be inter- 

descnption, claims, and drawings, in various combinations connected so as to form a meshed network (e.g., a partial or 

or subcombinations. It will be apparent to those skilled in fully meshed network). Additionally, the wide area network 

network theory and design, in light of the present (WAN) 1 may contain any number of IP switches 502 

specification, that alternate combinations of aspects of the s located within the WAN 1 such that it is not connected 

invention, either alone or in combination with one or more directly to any endpoint routers 919, and/or one or more IP 

elements or steps defined herein, may be utilized as modi- switches 502 may be located at an interface between the 

fications or alterations of the invention or as part of the SPN 500 and 40 endpoint router 919. In further embodi- 

invention. It is intended that the written description of the ments of the invention, there may be multiple endpoint 

invention contained herein covers all such modifications and 10 routers 919 associated with a UNI 402/IP switch 502 and/or 

alterations. multiple UNIs 402/IP switches 502 associated with an 

endpoint router 919. 

BRIEF DESCRIPTION OF THE DRAWINGS The network architecture of the WAN 1 allows the num- 

■ni f fnir»mna„ m n,. f ,u • n . L ber of IP switches to increase as customers are transitioned 

H S ? ?Z mmar ?.° f *° "™ ntl ° n . « ■» ^ to m e new service. For ex le> „ shown in FIG g . 

showr, tZ g ^ m °T^ ^ °, Tf m from non - IP enabled "NIs 402 (e g , S 

shown m the drawing* These exemplary embodiments, A) may be routed to „ , p switch 502 ei^h^e in ,he 

however, arc not intended to limit the invention solely M n ; lm „,i A i.h„,,„h ,u; „ , , ^.V 1 ■ 

thgjgjQ ' network. Although this creates some negligible rnefficien- 

' , .„ cies in "backtracking" it nonetheless allows a migration path 

FIG. 1 illustrates a wade area network (WAN) having to the new network architecture without simultaneously 
routers as CPEs and PVCs between customer locations. replacing all routers 501. However, as more and more users 

FIG. 2 shows data flow through the WAN shown in FIG. 25 are tra nsitioned to the new network architecture of WAN 1, 
1- more and more IP switches can be added (FIG. 8B) to 

FIGS. 3-5 show the construction and flow of data packets accommodate the increased load. In many embodiments, it 
through the network. may be desirable to eventually convert each UNI. 402 to an 

FIG. 6 shows a block diagram of a network architecture ^ SW j tch 502 such that IP routin S may be accomplished at . 
in accordance with aspects of the present invention. 30 me edge of ^ network - 

FIG. 7 shows a detailed block diagram of the network u- Id some ^bodiments the WAN 1 may include a com- 
ilhistrated in FIG 6 bination of conventional network switches and/or routers 

Pin ra_*r ch mm • ,• ,u f • 501 in addition to IP switches 502. On the other hand, every 

Z' ftf migration path for incorporating switch in the SPN 500 may be an IP switch 502 

aspects of the invention into conventional network architec- 3S Alternatively, the WAN 1 may contain only a single IP 
' switch 502. The IP switches 502 may be variously config- 

FIG. 9 shows data flow through the network architecture ured to include a suitable multi-layer routing switch such as 
of FIG. 6. a Tag Switch from Cisco. Multi layer routing switches may 

FIG. 10 shows application based prioritization through also be utilized from vendors such as Ipsilon, Toshiba, IBM, 
the network architecture of FIG. 6. 40 and/or Telecom. IP switches are currently being developed 

FIG. 11 illustrates an exemplary embodiment of a means to rcplace en dpoint routers so that customer premise equip- 
to apportion services through the network of FIG 6 mcnt ( cg '' Etflemet l° ca l area network WAN) equipment) 

FIGS. 12-14 illustrate data flow through exemplary /l^"'?' f*?* * aS y nchrOQOUS ,ransfer m ° dc 
WANsl. (AI M) network. Aspects of the present invention propose 

45 "sing IP switches in a different manner to maintain the huge 
DETAILED DESCRIPTION OF PREFERRED installed base of customer premise equipment while avoid- 

EMBODIMENTS mg . limitations of previous systems. Accordingly, the IP 

switches in accordance with embodiments of the invention 

Exemplary embodiments of the present invention allow are disposed within the SPN 500 and modified to provide 
the large installed base of frame relay customer premises so suitable routing and interface functions 
equipment (CPE) to be maintained by using the same In some embodiments of the invention, an IP switch 502 
interface in a different way to deliver new sets of services acts as a multi-layer switch. For example, an IP switch 502 
and features to the customer For example, the data link may receive ATM cells, switching some or all of the ATM 
connection identifier (DLCT) known from the frame relay cells based upon the content of IP packets encapsulated 
protocol may be used to select among several virtual private 55 within the ATM cells. Thus, IP addressing may be used bv 
networks with differing address spaces, feature sets, and/or an IP switch 502 to determine an ATM virtual path for 
conventional permanent virtual circuits (PVCs). sending ATM cells to a destination UNI 402. In further 

Referring to FIG. 7, a block diagram of a wide area embodiments of the invention, higher layer addressing (e.g., 
network (WAN) 1 incorporating aspects of the present transmission control program (TCP) logical ports at layer 4) 
invention is shown. The WAN 1 includes a plurality of 60 may also be used by an IP switch 502 as a basis for switching 
customer premise equipment (CPE) system, for example ATM cells to provide a path through the SPN 500. In still 
routers located at each of the end user locations and inter- further embodiments of the invention, an IP switch 502 uses 
connected via one or more service provider's networks IP addresses and/or TCP logical ports to make quality of 
(SPNs) 500. The SPN 500 is typically connected to a service (QOS) decisions. 

plurality of endpoint routers 919 via a plurality of corre- 65 In further embodiments of the invention, an endpoint 
spending user network interfaces (UNIs) 402 and/or one or router 919 may encapsulate one or more IP packets in frame 
more internet protocol (IP) switches 502. The IP switches relay frame 914. In this event, the frame relay frames may 
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be transmitted between an endpoint router 919 and a corre- 3 and 4 about which virtual private network (VPN) service 

spending UNI 402 and/or IP switch 502. The endpoint router class, or conventional P VC the packet should be routed to 

919 encapsulates IP packets 950 with frame relay frames Thus, a packet with layer 4 information indicating it is a 

914 Further, the endpoint router 919 may set the DLCI of telnet (interactive) application and layer 3 information that 
each frame relay frame 914 according to a particular service 5 it is an internal company address might go to VPN A for a 

category (if a service category DLCI is used) that the user low-delay intranet class of service. Another packet that is 

has selected. For example, the various service categories part of a file transfer protocol (FTP) file transfer might go to 

may include the public internet, communication via a local VPN B with a lower service class, and a third packed going 

c— C °T Un T 0n W i hm 7 l0Sed USCr 8rP< CUG >; between two heavily utilized applications might go on a 

communication with an extianet (e.g., a network of trusted ,„ dedicated PVC D. These decisions are coded as liferent 

suppliers or corporate trading partners), ive audio/video DLCI values, inserted in the layer 2 frame, and sent into the 

transmission, multicasting, telephony over internet protocol UNI 

(IP), or any combination thereof. Thus, the concept of a aI.uiihia/im ,u . _, . ^ T ™ , 

frame relay PVC is significantly expanded by aspect of the %, UN1 ^ ^ ° D °*l ^ ^ 

present invention. Forexample; the locationof an intended „ ^ ^^PnZ^ T, v° h ^ 2 ^ 

network endpoint recipient is not necessarily determined by 15 ^ ° nff , f J ^ u ^™ ?T 

a DLCI at the endpoint routers 919 stripped off as it is forwarded to VPN A. Within VPN A, the 

r„ f, „„u a ' . r .u ■ „„„ .„„ layer 3 address is now used to make routing decisions that 

In filler embodiments of the mvenuon, a UNI 402 may send the acket t0 its destiaation UNI . ^ no pvc need 

l»4 Z£T 'h " &am T, 14 ff ° m iD r dP f ° mt r0UtM 9 f 1? be cstablkhed ^ ° f ^ for «>d conventional 

and divides and encapsulates frame relay frames into, for M routing methods and protocols can be used, as well as newer 
example, smaller fixed-length , ATM cells. The UNI 402 may routing techniques. This permits VPN A to 

'I ^ In H T X rt V mt °,M ^ ^ P TOvide a ° f connectivity between sites wutut 

vnf« ^ P idenufier/virtual channel identifier (VPI/ requiring the customer to configure and maintain the "mesh" 

IT™* ^, h J° r T example > ^ Network 25 B is treated similarly except that VPN B is implemented 

L^f^ R ^f^i m Imple T n,a f n . Agree " ^ a lowcr service class (e.g. higher delay). Finally, the 

tT™ v « hT h « y ^T' ^ SemC6 P*** forwarded to PVC D ^ its layer 2 frame intact and 

S £ md " , £f d ^ ned m Im P u lementa ? n P*^ 'n^gh the network as a conventional frame relay - 

ment#8ofmeFrameRelayFommmaybeutil 1 zed.AnATM frame . This dlows customers to maintain their current 

address associated with a service category DLCIs defines an 30 connectivity of PVCs for their high utilization *aL^ns 

ATM yirtual Path via network routers to an IP switch 502. but still nave a high mesh of co ^ ecdvily ^ 

Thus, ATM data associated with a service category DLCI is VPNs 

ultimately sent to an IP switch 502. However, ATM data n. ' - - , ■ ■ 

associateUmaconventionalDLCImayormayn^tbesent , ^iZ'Z™ T** *'™«>™> [ he WAN 1 
to an IP switch 502 and may be routed through the network « S? ? 5 °° f My * mtable faSt pa ? et network 

without nassine through ,„ TP *w^h srrf tw 35 reiving frame relay data packets havmg user data in a user 



without passing through an IP switch 502. Thus, both Z« h 7 w/vT^^jfu • 

translated IP data and conventional PVC data may be present f* 4 . 6 ^ or ?PN MO then switches 

in the SPN 500 and/or WAN 1 P cketS 1151118 0ne or more IP s^nes 502 responsive to the 

in f,,rti„.r .o r',u • .• t ixrr fota. The user data may be used to discriminate between 

» t ^^^f ™ff™> a UNI 402 , plurality of categories based on user 

£h IP ^ T rt r nd d K a , t ° 3 PrCd F Ct r 40 data - RoutiD g over WAN 1 ««Vb' SPN 500 may be 

Mention TlSlS h/" k f"* 0 * 10 ??!* °f ^ responsive to at least one of the different service categories 
3 S ^-2Si?f ? , roUtCr 5 °f ^ includin S d ^criminating based on multicast data. 

^ hLT ^ i , k fl baSC u d UP f aD a i?° nthm , Additionally, the WAN may generate a fast packet address 
ocMion^f TP °, h ^ C ,K° WS ' the / e ' aaV K e dls,anoe/ field responsive to the IPpacket data and route the IPpacket 
S h i T i' W °^ l \ bemg ^-' 45 fcough me fast packet network responsive to the fast Jacket 
™* o7,h ^ ^ 1V° St,U £ i rther ^cnt address field " Further > laver 4 inforniation may be utilLd to 
Ltr 5 ^, 7™°* 3 ?,k ' ne T 0lk r ° UtCr I"' determine ^ " uaIit y of service - ^ ^ a ^y »f service may 
one UNI 402, network router 501, and/or IP switch 502, information rate, priority information, delay, loss, 
depending upon, for example, a service category or catego- S0 availability, etc. Security features may be taplemented in 
, , . the IP switch such that routing tables for each of the users 

In further embodiments of the invention, a UNI 402, an IP are separated based on one or more service categories and/or 

wwri^n^ 3 hT ™° 1 ,? 5mpareS M ^ users " In ,his manner svstem 18 ™ d * m ore secure. Still 

VPVVCI 303-305 address with an IP address for the same further, the system may receive a plurality of frame relay 

data. If the two addresses are inconsistent, then the ATM cell ss packets over a permanent virtual circuit (PVC) at a first node 

may be discarded, sent to a pre-determined address, and/or in an asynchronous transfer mode (ATM) network, generate 

returned to the sending location. In even further embodi- an ATM address based on a data field other than a data link 

ments of the invention, layers above the layer 3 IP layer may connection identifier (DLCI) within the frame relay packets 

be used for address and/or service class generation/ and then route the packets through the ATM network based 

discrimination. For example layer 4 of the ISO addressing 60 on the ATM address. The routing of packets may be respon- 

scheme and/or alter application level data may be utilized to sive to one of a plurality of service categories. The system 

determine particular service classes. may provide routing ubles aj] ^ 

Referring specincaly to FIG. 9, the path of user data for each of a plurality of different service categories The 
flowing through an exemplary WAN 1 is shown. As in the different service categories may be determined using inter- 
frame relay case, user data at the application layer and layer 65 net protocol (IP) data within a data field of a packet passed 
4 requires the addition of a layer 3 network address header. by the ATM switch. In a fast packet network a fast packet 
In the CPE a decision is made based on information in layers switch may compare an address of a fast packet with a layer 
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l^^V/ZT" 1 - ^T* <?° ntained wWlin ,he fast element 1004 and switch 1003 are connected to Customer 
packet and determining whether the fas. packet address is Site A via frame relay switch 1001. Routing elemem MW 

consent wuh the layer 3 IP address. Further, for security, and switch 1006 are connected to OistomeTfite E TvTa frame 

exlT^ionT ^ tC Pr ° Vided f ° r ^ swi,ch 1009 - R ° utin § 1012 and sw^ch iwl 

SeT * W?;' 4 desUnaUon addres «- s are connected to CustomTr Site C via frame relay switch 

S SSed -rtfwA^ rrv 1 "^ H ^ toC ° a - 1016 - R0UtiDg element 1013 ^ M 1015 are c °^d 

^ISS^W^ 10 a V 0 %%??- ^ frame r SWitCh 1017 ' lD ^ 

mode (ATM) switch coupled to and re^vSm the CPE ^ ^ T° T 6 ^ 10 °° & ° m CffilM£ ' 

frame relay data packets' and including traSation f * Th « T 3 'T' ? ° L ? WN 

circuitry for translating data link connection identifiers from 10 ££?JK T "* ' 3 addreSS P ointin S 

the frame relay data packets into ATM addresses represent- ??T *te R In SUch c a Case ' frame rela y 1001 

ing a plurality of virtual private networks basld oT a ^1 U hT ^ ' ^ ^ ^ 1002 «° ^ 

predetermined service catego'ry associated with a particular J2f i.^ft ^ K elemen, l . 1004 associaled 

DLCI; or the WAN 1 may include customer prerSse" ^ ,L ^ by SW1 ' ch 1003 ' ,he &ame 

equipment (CPE) and a fast packet switch coupleTto *e 15 fJZ^ l°T\ l™ 04 ™ 1 ** ^..ncnts short-cut 

CPE via one or more permanent virtual circuits and receiv! ™ ?h f ^ r ^ ^ r0Uter/switch 1003 ' ™<» 

ing frame relay data packets, the fast packet switch including ^? ^t^l^ZX!^ '° d ^ f mmate . between differ- 

address translation circuitry for translating user data within h^Th S^T? ,nformaUon mav ^ be 

the frame relay data packets into fast packet addresseT ^carded Next, the layer 3 information in combination with 

In embodiments of the present invention data security is M * ^ '° * T*" 8 decisi0n - In 

enhanced in that data may be easily accuratel^Sed t ' f rou ^S decislon «™ ld ™* a layer3 PDU 1011 

for inconsistencies a. ^destination Th s"" ^ ca^J tfese 10U Si^eT 1 '^^'^ T' i"" 7 ^ ^ 3 

embodiments operate using both layer 2 and layer 3 fW ,n V e h nca P sula « ed a ^ 2 frame, the 

DLCI of the narWpt ranH ihi.c ;r,f«^,,r» u . .u ' inc. frame relay frame 1010 is then delivered 

i7u , u \™ mfomation about the source 30 to the Customer B router, 

of the packet), the VPN processor may cross-check the icl L„ r ■ , ,. , 

DLCI with the source IP address in the packet to see if Z vr£ , T °* for the " 

source IP address is in the range known from the orfgUtnig ™ ! ^ ^ to ^ 

site. Thus, the problem associated with the spooZ oHP T£ ^ Z^^- PKSCat * ^ Switcbin S 

source addresses may be significantly reduced ^ aS , Sh ° W ° 1 ™ FIG ' 13 " ^ can reduce me bac ^l 
In still further embodiment of the invention' a UNI 402 " n^°f J n r * gCt ? thC - routw/switch P^cessing 

an IP .vitch 502, and/or a network roler ?S hL^JS n^iffl^XfSSS? ^£^1 ^ " 

and/or partitioned routing look-up tables. Routing Lies S^^^nSiS SnSS^ 1 

Uble containing the comer's ,P network address inj ^^I?«2^S^l^i 1 S B eS 

sour^ ofYfrTm^te^CI maTh" "T^ *" ^^#1 viaswitching node. 1501, SoSd £aS 

IP^Ttrh nf^, v DLCI may be used as an index by an 1504 would be received at Customer Site B. 

Ir switch, network router, and/or UNI for determinine which t , ,. m . 

routing table to use. This allows customersToTve lefr 45 fo ^» m « em bod.men te , an ATM core network may be used 

routing table size and speed governed by their individual tl f ^ ^ mterfa0eS may be ^ to 

address space, thus speeding the routing processTr^de - ^1 T 1116 "l" eXCmplary embodim «' 

ably. The use of septate routing table! also proWfe an ^ ^ ZT* K ShOWD " HG - 14 " In ^ 
added measure of security, as packets cLofbe'm^route" so SSoSSTSnT? t STT,™ C ~ Bd 10 

due to errors or updates in routing information related to Cus,omer Site A via switch 2000 and a frame relay/ATM 

other customers. niormation related to conversion unit 2001. Switch 2019 and router 2018 are 



CIU>IU1L1 CIS. ****** *.viu ai& 

r • - # connected to Customer Site B via switch 2005 anH framp 

^s-=s ^ ~ ~ - 

«dy. This „y b. b, ElTofMp™. SL i ' , * ^""^ S, "« 

EST ' '"^ " d " S= ° ™ SSSSK =SS 

inr- t> n . . . mav be 56,11 down 111 A™ PVC designated for VPN #1 

FIG. 12 illustrates an exemplary WAN 1 having both 65 processing. ATM cells 2002 may then be forwarded to 

conventual routers and IP switches incorporating aspects switch 2003 and router/switch 2004 (which may bT^tach^ 

of the mvention. In this exemplary WAN 1, a routing to switch 2003), where the ATM ceUs may be re^nS 
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to obtain the layer 3 packet information for routing within RgCDR if 
VPN #1. Once the address information has been extracted 

from the layer 3 packet, the packet may be segmented again „ 
into ATM cells 2009 that can be transferred through the L J i sCM 
network. After being sent through router/switch 2018, 2019, 5 ' 
ATM cells 2008 may be converted from cells to frames at the 
external conversion unit 2006 and switch 2005. Customer 

Site B would then receive frame relay frames 2021 . Thus, an v 
extra segmentation and reassembly (SAR) cycle may be ff = Z, Ji 

required when using an ATM backbone with a core of ,„ ' 
router/switches. However, if the VPN processing is pushed 
outward to edge switches, the extra SAR cycle may be otherwise. 

eliminated. The extra SAR cycle may be eliminated because If the aggregate offered traffic rate ZS,. does not exceed the 
conversion from frame relay frames to ATM cells may take CD R, then 100% of the offered traffic from each source i 
place in the same unit where VPN routing decisions are may be delivered through the WAN 1 and/or SPN 500 to the 
made. 15 destination. However, when the aggregate offered traffic rate 

Traffic management may be variously configured in the ' h f ^^l™ 1 ^ f N ^ have 

WANland/ortheSPNS00 y Forexample!froma^tomer^s ?&Jd^ to fi^^" de ^ nte of 

JKttrilS? SPN 500 may ensure certain S r I ! ■ - £55 

traffic rates tor the customer. 20 rate of traffic delivery R to a destination -is at least equal to 
in a network, data traffic may be sent from multiple the destination's assigned CDR. In the situation where R is 
sources Jo a single destination (multi-point to point). A reduced by the network, it may be desirable to enforce 
"source" is defined as the user -transmitting side of, for "fairness" for each source. In other words, it may be 
example, a UNI (i.e., the customer side of a UNI, which may ' desirable to ensure that no single source may be allowed to 
be external to a WAN and/or to a VPN), a switch, an IP 25 be greedy by obtaining a disproportionate amount of net- 
switch, and/or a router at or near the edge of a network. A work bandwidth at the expense of other sources, 
"destination" is defined as the user receiving side of, for To P rav ide for fair access to the WAN 1 and/or SPN 500, 
example, a UNI (i.e., the network side of a UNI), a switch, m s ? mc embodiments each source is assigned at least one 
an IP switch, and/or router at or near the edge of a network' destination rate share (DRS). A DRS is a rate, measured in 
Traffic that is offered for transmission by a source to the 30 data units per unit of time (e.g., bits per second). A separate 
WAN 1 and/or SPN 500 is defined as the "offered traffic " DRS and/or 1x1 of DRSs ma y be assigned to each source 
Further, a "VPN source" and a "VPN destination" are a /or gr0Up ° f so" 1 "**- Further, the DRS or DRSs for a 
source and destination, respectively, which belone to a aiven § 1V f D S ? UICe J 03 ^ de P en d upon the destination or set of 
VPN. Agiven UNI, if simultaneously sending andreceiVing h ^ the , son «« ""X ^affic to. In other 
may simultaneously be a source and \ destina^! 35 ^^^TSL DRS* Tt'' ^ ^ ^ 
Furthermore, a given source may offer data traffic to mul- J k ,he , DRS , as ! 1 f ed between a source 1 and 
tiple destinations, and a given destination may receive traffic 1^^^ ™* °f d ~ on ?>- us ' "» *»» 
from multiple sources embodiments, the DRS may be different for a given source 
• in cnm* Pm K„ri; m o„.o t .u ■ i • j depending upon which destination it is sending traffic to. In 
deLw «te fSS^iv he m lT° n U N C ° mmi " ed farther sediments, the DRS for a given source may be 
7^,e rn R c H ( fi h } T aSSlgned l to ea ? des ^aaon. 40 constant, independent of the destination. 
ineLDK is denned as the average number of bits per second wi>»„ . ™ -it . ^ 

average" will be used throughout, any other similar algo- 45 share » of ^ destmaU^nTcDR £ defined asT sources 

nthm may be used, such as the mean, the sum, or any other DRS divided by the aggregate DRS of Tcdve sZLs 

useful measurement and/or statistical calculation. If the transmitting to a given^fdna.^f Thus elch acti^ 

averagerateofag g regateofferedtraffic(i.e.thetotal offered fa f r share ^ of ^defined Lthe 

traffic) from one or more sources to a given destination is following: oennea as me 

greater than or equal to a given destination's assigned CDR, 50 

then the WAN 1 and/or SPN 500 may guarantee to deliver DRS , 

traffic addressed to the destination at an average rate equal n = ' cdr. 

to or greater than the CDR. If the average rate of aggregate f 
offered traffic is less than the CDR, then the WAN 1 and/or 

SPN 500 may deliver the offered traffic to the destination at 55 -n,» ,,.„, 1 ^, 1 . ■ ~ L 

the aggregate offered traffic rate (100% of the offered irtfLJ a J? nr IZ *\Z IT* transn »^ 10a rate - \^ ^ WAN 1 

To clarify, let the number of active sources sending traffic to 11^1^ l l & ** COnf ° rn,m ? btBi f guaranteed to 

a particular destination be N. As will be described in more L.iS Aelltl. * ^ ™ V 

detail below, a source may be considered "active" during a ^ following, 

given time window if the source offers at least a threshold 60 

amount of traffic to the WAN 1 and/or SPN 500 within the 

given time window. Let S, be the average offered traffic rate, Ta S '* cdr 

or "offering rate," from each source i toward a single given ' 
destination, wherein i-[l, . . . , N]. Further, let R be the total 

rate at which the WAN 1 and/or SPN 500 actually delivers 65 T 5min(r S ) 

wiU^ 2 d h eS 1 tinati0,L Th£n ' th£ WAN 1 "M™ SPN 500 n»us, in these embodiments the WAN 1 and/or SPN 500 

wi prow a . may enforce fairness by reducing one or more sources' 
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actual network transmission rate T, at most from S,. to r< of reference rale B effectively reserves network bandwidth 
ensuring that each source obtains its fair share of the CDR. for an inactive source, thus ensuring that a previously 

wfSiT ^ ™m aChieVC \? te ° f 31 lcaSt CDR ' the mactivc "»» ionics active can send at least some 

WAN 1 and/or SPN 500 may at its discretion transmit traffic traffic through the network during time period W 
from a given active source or sources at a rate greater than s Specifically, the WAN 1 and/or SPN 500 may ensure that 
r,.. In fact the WAN 1 and/or SPN 500 may at its discretion each source's T, is guaranteed to be at least a minimum 
transmit data from a source i at any rate between and reference rate B. In this situation, a source is considered 
inchjdmg the fair share rate r, and the full offered rate S,.. active during W, if more thanB*W,units of data (e e bits} 
J„V g ?, ate eDxf cIS 3 S ? urce „ ma y te considered by the are received during W y . It is desirable to define B to be 
WAN 1 and/or SPN 500 to be a non-conforming source." relatively small as compared with S,. so as to retain as much 
Conformance of a source may be calculated using a standard excess bandwidth as possible, yet still large enough to 
leaky bucket algorithm with variable drain rate. Thus, the ensure network availability to a non-active source (non- 
conforming depth' of a "bucket" would be DRS.-W. In sending source with respect to a given destination) that may 
other words, the maximum number of bits that will be sent later become active with respect to a given destination In 
to the network within a given time window of length W is some embodiments, B may be a predetermined rate In 
equal to DRS,*W. During a given time window of length W, 1S further embodiments, B may vary with time, with the 
the "drain rate" of the "bucket" is equal to T, which is number of inactive sources, with the number of active 
calculated during previous time windows. Thus, date pack- sources, and/or with the total number of sources. In still 
ets inserted "above" the conforming bucket depth may be further embodiments, B for a source may depend upon a 
labeled as a "non-conforming." In other words, for a given priority classification assigned to the source. In still further 
time window, data packets in excess of the total DRS ; *W 20 embodiments, when a previously inactive source becomes 
number of bits may be labeled as non-conforming data active, the priority assigned to the source may depend upon 
packets. In such a situation, some or all of the source data me content of the data (e.g., data payload, DLCI, and/or 
packets equal to the difference between S, and T,- may be address) offered to be sent. Thus, B may not be the same for 
labeled as non-conforming data packets, and some or all of eacn source. 

the non-conforming data packets may be dropped. 25 0nce the excess bandwidth is determined, the maximum 

This does not mean that data cannot be of a bursty or conforming actual network transmission rates, T„ may be 
rate-variant nature. Although exemplary embodiments have calculated. To accomplish this, T, for each source may first 
been described as operating using average rates, real-time be xt bv default to minfr,-, Sj. Then the excess bandwidth, 
rates may vary within any given time window of length W. E > mav be distributed among some or all of the sources that 
Thus, a certain amount of burstiness of data is allowable. 30 are actively transmitting to the given destination, thus 
This maximum burst size is the maximum number of bits adjusting or raising T, for these sources. In some 
that the WAN 1 and/or SPN 500 guarantees to transfer embodiments, the excess bandwidth may be uniformly dis- 
during a time window W. tributed among some or all of the active sources. In further 

In further embodiments of the invention, the WAN 1 embodiments, the excess bandwidth may be distributed 
and/or SPN 500 may provide forward congestion notifica- 35 among these sources according to source priority, data 
tion to a destination. For example, the WAN 1 and/or SPN priority, and/or DLCI. 

500 may provide a layer 2 binary indication that the CDR is In runner embodiments, the WAN 1 and/or SPN 500 may 
being exceeded by using the frame relay forward explicit provide backward congestion notification to a non- 
congestion notification (FECN) bit and/or a layer 3 message conforming source. Such notification may be in the form of 
that indicates a non-conforming source and optionally con- 40 a laver 2 and/or a layer 3 message indicating a destinations) 
tains rate information for that source (e.g. the actual trans- *° r wn ich the non-conforming source is exceeding T f and/or 
milted rate T f and/or the excess rate S.-T^. Furthermore, in rate information for the non-conforming source (e.g. the 
some embodiments, multiple non-conforming sources might actual transmitted rate T,- and/or the excess rate S.-T^). 
be listed, even within a single message. In these forward However, a layer 2 notification by itself may not be 
congestion notification embodiments, conformance may be 45 preferable, since a source receiving only a layer 2 notifica- 
measured at the network side of a destination. In some non may not be able to distinguish between destinations to 
embodiments, a forward congestion notification may be which the source is conforming and those for which it is not 
provided to a given destination when the offering rate S,- of conforming. In some embodiments, a backward congestion 
an active source offering to send traffic to the destination notification may be provided to a given active source when 
exceeds the actual network transmission rate T,. for the 50 me offering rate S ; of the source exceeds the actual network 
source. ' transmission rate T,- for the source. In further embodiments, 

Non-conforming packets that cannot be transmitted on the a user at a non-conforming source may be notified of 
egress port of a source may be dropped with or without any congestion information, the assigned CDR, DRS^ r^ and/or 
indication to the source or destination. To measure conform- ' n st ^ further embodiments, it may be up to a user to 

ance of a source, the amount of excess bandwidth available S5 decide how to act upon a congestion notification. In even 
to the sources for transmission to the destination should be further embodiments, a source may reduce its offering rate 
determined. To calculate the excess bandwidth, let W - be the S ' m response to receiving a backward congestion notifica- 
j'* time window. The excess bandwidth above the fair share non - 

bandwidth may be computed as ln these backward congestion notification embodiments, 

60 conformance may be implemented at the network side of the 
E=CDR- V mhtr, S)-mb source VN1 - ln such embodiments, feedback concerning the 

i ' destination delivery rate may be required from the destina- 

tion. The feedback may also contain information regarding 

. the rate share of the active sources at the destination and/or 

wherein M is defined as the number of possible sources from 65 the CDR divided by the aggregate rate 

which a destination may receive traffic, and wherein B is While exemplary systems and methods embodying the 

defined as a predetermined reference rate. The introduction present invention are shown by way of example, it will be 
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understood, of course, that the invention is not limited to 
these embodiments. Modifications may be made by those 
skilled in the art, particularly in light of the foregoing 
teachings. For example, each of the elements of the afore- 
mentioned embodiments may be utilized alone or in com- s 
bination with elements of the other embodiments. 
Additionally, although a meshed network is shown in the 
examples, the inventions defined by the appended claims is 
not necessarily so limited. Further, the IP switch may 
convert from any higher level IP like protocol to any 10 
fast-packet like protocol and is not necessarily limited to the 
ATM/IP example provided above. Furthermore, examples of 
steps that may be performed in the implementation of 
various aspects of the invention are described in conjunction 
with the example of a physical embodiment as illustrated in is 
FIG. 5. However, steps in implementing the method of the 
invention are not limited thereto. Additionally, although the 
examples have been derived using the IP protocol for layer 
three, it will be apparent to those skilled in the art that any 
version of IP or IPX could be used as the layer three 20 
routeable protocol. Furthermore, it will be understood that 
while some examples of implementations are discussed 
above regarding IP and ATM protocols, the invention is not 
intended to be limited solely thereto, and other protocols that 
are compatible with aspects of the invention may be used as 25 
well. 

What is claimed is: 

1. A method comprising the steps of: 

receiving into a fast packet network frame relay data 

packets, said frame relay data packets having user data 30 

in a user data field; 

switching said frame relay data packets within the fast 
packet network responsive to the user data, wherein the 
user data includes an internet protocol packet; 
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generating a fast packet address field responsive to inter- 
net protocol packet data; and 

routing the internet protocol packet through the fast 
packet network responsive to the fast packet address 
field. 

2.1hc method of claim 1 wherein the step of generating 
the fast packet address field occurs in a node located at an 
edge of the fast packet network. 

3. The method of claim 1 wherein the step of generating 
the fast packet address field includes routing the internet 
protocol packet data within the fast packet network to a node 
capable of generating the fast packet address field respon- 
sive to the internet protocol packet data. 

4. The method of claim 1 wherein the fast packet address 
is generated at a single node within the fast packet network. 

5. The method of claim 1 wherein the fast packet network 
includes a plurality of nodes capable of generating the fast 
packet address field responsive to the internet protocol 
packet data and nodes not capable of generating the fast 
packet address field responsive to the internet protocol 
packet data. 

6. The method of claim 1 wherein layer 3 data within the 
internet protocol packet data is utilized to generate the fast 
packet address field. 

7. The method of claim 1 wherein layer 4 data within the 
internet protocol packet data is utilized to generate the fast 
packet address field. 

8. The method of claim 7 wherein the layer 4 information 
is utilized to determine a quality of service. 

9. The method of claim 8 wherein the quality of service 
includes an information rate. 

10. The method of claim 8 wherein the quality of service 
includes priority information. 
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It is certified that error appears in the above-identified patent and that said Letters Patent is 
hereby corrected as shown below: 



IN THE DRAWINGS : 

Fig. 9, delete "PCV D" and insert - PVC D --. 
Column 1 . 

Line 32, delete "(AIM" and insert - (ATM) -. 
Line 55, delete "(PDI)" and insert - (PDU) -. 

Column 2 . 

line 39, delete "is" and insert — are --. 
Column 3 . 

Line 60, delete "thereof Thus", and insert ~ thereof. Thus, --. 
Column 4 . 

Line 57, delete "UN," and insert - UNI, -. 
Column 5 . 

Line 61, delete "premise equipment (CPE) system," and insert — premises 
equipment (CPE), — . 

Column 6. 

Line 41, delete "premise" and insert — premises -. 
Line 42, delete "WAN)" and insert -- (LAN) --. 
Line 46, delete "premise" and insert — premises — . 
Line 67, delete "frame" and insert — frames — . 

Column 7 . 

Line 22, delete "translates" and insert - translate ~. 
Line 30, delete "DLCIs" and insert - DLCI -. 
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